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CETA  Demonstration  Provides 
Lessons  On  implementing 
Youth  Programs* 


The  Youth  Incentive  Entitlement  Pilot  Proj¬ 
ects  was  designed  to  test  whether  guaranteed 
part-time  jobs  during  the  school  year  and  full¬ 
time  jobs  in  the  summer  would  induce  eco¬ 
nomically  disadvantaged  youths  to  remain  in 
or  return  to  school.  GAO  found  that  most 
worksites  provided  adequate  experiences. 
Others,  however,  had  problems  in  development 
of  good  work  habits,  in  supervision,  and  in 
monitoring  and  couseling.  Also  many  sponsors 
experienced  operational  problems. 

The  Department  of  Labor  and  its  contractor 
had  to  intervene  frequently  in  program  oper¬ 
ations.  This  raised  the  questions  of  whether  the 
demonstration  was  a  fair  test  of  the  guaranteed 
job  concept  and  whether  sponsors  are  capable 
of  implementing  complex  programs. 

GAO  recommends  that,  if  the  Congress  decides 
to  expand  the  Pilot  Projects  or  undertake  a 
similar  program,  it  do  so  gradually  to  allow 
sponsors  time  to  develop  the  capacity  to  prop¬ 
erly  implement  the  program. 
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To  the  President  of  the  Senate  and  the 
Speaker  of  the  House  of  Representatives 

This  report  describes  youths'  work  experiences  in  one  of 
the  largest  demonstration  programs  the  Federal  Government  has 
ever  attempted  in  the  field  of  youth  employment.  In  addi¬ 
tion,  it  describes  serious  operational  problems  encountered 
in  implementing  the  demonstration  and  questions  whether  the 
demonstration  was  conducted  in  a  real-world  environment.  We 
made  this  review  to  give  the  Congress  information  to  help  it 
develop  a  sound  long-range  strategy  for  dealing  with  youth 
unemployment . 

We  are  sending  copies  of  this  report  to  the  Director, 
office  of  Management  and  Budget,  and  to  the  Secretary  of 
Labor .  A 

Comptroller  General 
of  the  United  States 
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COMPTROLLER  GENERAL'S 
REPORT  TO  THE  CONGRESS 


CETA  DEMONSTRATION  PROVIDES 
LESSONS  ON  IMPLEMENTING 
YOUTH  PROGRAMS 


D  I  G  E  S 


T 


The  Department  of  Labor's  Youth  Incentive 
Entitlement  Pilot  Projects  (Entitlement) 
is  a  $222  million  demonstration  designed  to 


^-test  whether  a  guaranteed  jobs  program 
would  induce  economically  disadvantaged 
youths  to  remain  in  or  return  to  school 
and 


^--provide  jobs  to  reduce  the  high  unemploy¬ 
ment  rate  among  these  youths . j 

Labor,  while  retaining  overall,,  responsibility 
for  the  program,  contracted  with  the  Manpower 
Demonstration  Research  Corporation  for  the 
research  design,  management,  and  evaluation 
of  the  demonstration.  (See  pp.  2  to  4.) 

o 

'  GAO  found  that: 


C I  J-^-Most  worksites  provided  adequate  work  ex¬ 
periences,  but  others  had  problems  in  job 
quality,  supervision,  job  development, 
counseling,  and  worksite  monitoring. 
iSee  pp.  10  to  24.)  r 


r*  ■ 

^  i  ’--Many  sponsors  experienced  serious  operational 
problems  implementing  or  complying  with  pro¬ 
gram  requirements^  (See  pp.  2&-to  38.) 

0-  4  5  ^  ^Labor  and  the  Corporation  frequently  inter¬ 
vened  in  sponsor  operations,  thus  raising  the 
questions  of  whether  the  demonstration  was  a 
fair  test  of  the  guaranteed  job  concept  and 
whether  sponsors  are  capable  of  implement¬ 
ing  complex  programs  such  as  Entitlement. 

(See  pp.  47  to  51 . ) 
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RECOMMENDATION  TO  THE  CONGRESS 


If  Entitlement  is  to  be  extended  or  used  as 
the  basis  for  future  youth  initiatives,  the 
Congress  should  recognize  the  conditions 
under  which  the  demonstration  was  imple¬ 
mented.  Certain  competitively  selected 
sponsors  receiving  intensive  oversight  had 
significant  operational  problems.  If  the 
Entitlement  concept  is  to  be  extended,  or 
expanded  to  other  sponsors  or  a  similar 
program  adopted,  GAO  recommends  that  the 
Congress  make  sure  it  is  implemented 
gradually  to  allow  sponsors  time  to  de¬ 
velop  management  capacity.  (See  p.  53.) 

MOST  WORKSITES  PROVIDED 
ADEQUATE  WORK  EXPERIENCES 

In  evaluating  the  quality  of  the  work 
experiences,  GAO  considered  whether  par¬ 
ticipants  exhibited  good  work  habits, 
received  enough  useful  work,  and  were 
adequately  supervised.  GAO  estimates  that 
81  percent  of  the  worksites  fostered  good 
work  habit  development,  73  percent  provided 
enough  useful  work,  and  74  percent  provided 
adequate  supervision.  However,  collectively, 
about  2,100  (40  percent)  of  the  5,200  work¬ 
sites  lacked  one  or  more  of  the  elements 
above.  (See  pp.  10  to  19.) 

Most  of  the  problems  at  worksites  are 
correctable.  Sites  evaluated  as  inade¬ 
quate  generally 

— had  too  many  youths  assigned  for  the  amount 
of  work; 

— did  not  adhere  to  time  and  attendance  pro¬ 
cedures; 

— had  work  of  marginal  value  to  the  employer, 
the  participant,  or  the  community; 

— did  not  enforce  work-hour  rules  and  allowed 
youths  to  be  paid  for  absences;  or 
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— had  unqualified  supervisors.  (See 
pp .  10  to  19 . ) 

Some  sponsors  had  difficulty  developing 
and  sustaining  a  sufficient  number  of  jobs 
in  the  right  locations,  particularly  during 
the  school  year.  Transportation,  school 
scheduling,  and  administrative  problems  con¬ 
tributed  to  this  situation.  (See  pp.  19 
to  21.) 

Sponsors  placed  a  low  priority  on  counsel¬ 
ing.  Most  counselors  spent  more  time  pick¬ 
ing  up  time  cards  and  delivering  paychecks 
than  counseling  youths.  (See  pp.  21  and  22.) 

Most  sponsors  did  little  worksite  monitoring 
during  the  first  year  of  the  demonstration. 
Monitoring  increased  during  the  second  year, 
but  some  problems  that  should  have  been 
obvious  were  not  always  identified.  Others, 
although  identified,  were  not  always  cor¬ 
rected.  (See  pp.  22  to  24.) 

OPERATIONAL  PROBLEMS  HAMPERED 
PROGRAM  PERFORMANCE 

Many  sponsors  had  persistent  operational 
problems.  Some  legislative  and  program 
requirements  were  largely  ignored;  others 
were  too  difficult  to  accomplish.  Some 
sponsors  rapidly  expanded  small  core  staffs 
with  inexperienced  personnel.  Staff  turn¬ 
over  was  high.  Ineffective  job  assignment, 
payroll,  and  management  information  systems 
contributed  to  major  reorganization  at 
some  sponsors.  (See  pp.  25  to  28.) 

Through  September  1979,  Entitlement  cost 
about  $4,600  a  year  per  participant.  Pro¬ 
gram  management  costs  represent  about  37 
percent  of  this  amount.  Rapid  program 
startup  combined  with  unrealistic  enroll¬ 
ment  projections  and  larger  than  needed 
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staffs  initially  caused  high  program 
management/participant  cost  ratios.  Less 
than  anticipated  enrollment  was  a  major 
cause  of  the  problem.  (See  pp.  28  to  31.) 

One  of  Entitlement's  major  features  was  that 
a  youth's  job  guarantee  was  conditional  on 
satisfactory  school  attendance  and  perform¬ 
ance.  This  feature  never  received  a  fair 
test  often  due  to  administrative  problems 
and  a  desire  not  to  reduce  already  low  en¬ 
rollments.  As  of  December  1979,  only  4  per¬ 
cent  of  34,000  youths  terminated  were  re¬ 
leased  for  unsatisfactory  school  performance. 
GAO's  review  showed  that  far  more  youths 
were  not  meeting  academic  or  attendance 
standards.  (See  pp.  31  to  33.) 

Sponsors  were  required  to  establish  relation¬ 
ships  with  other  groups  which  could  help 
them  serve  youths.  These  included  school 
systems,  community-based  organizations,  and 
other  organizations  serving  youths.  The 
results  were  mixed.  Some  sound  relationships 
were  developed;  others  deteriorated  badly 
over  time.  (See  pp.  33  to  35.) 

One  of  Entitlement's  innovative  features  was 
the  authorization  of  subsidies  up  to  100  per¬ 
cent  for  salaries  paid  to  participants  as  an 
inducement  to  private  employers  for  providing 
youths  with  jobs  and  for  testing  various 
subsidy  levels.  Many  private  sector  jobs 
were  with  employers  that  normally  hire  young 
workers.  GAO  recognizes  some  substitution 
is  inevitable  in  a  job  creation  program 
as  large  as  Entitlement;  however,  where 
substitution  occurs,  the  job  creation  impact 
is  minimized.  (See  pp.  35  to  38.) 

INTENSIVE  OVERSIGHT  REQUIRED 

To  respond  to  questions  raised  by  the  Congress, 
Labor  engaged  the  Corporation  which  designed 
a  three-part  research  program  to  analyze 


Entitlement's  impact,  implementation,  and 
cost.  (See  pp.  39  to  42.) 

Despite  intensive  efforts  by  the  Corporation 
and  its  consultants,  data  problems  persisted 
from  the  onset  of  the  demonstration.  The 
Corporation  is  confident,  however,  that  its 
cleanup  efforts  have  resulted  in  reliable 
research  data.  (See  pp.  43  to  47.) 

The  Corporation  had  to  intercede  in  program 
operations  at  many  locations.  Significant 
improvements  were  made  at  some  locations 
which  might  not  have  occurred  in  a  normal 
program  environment.  GAO  believes  that  Labor 
and  the  Corporation,  by  providing  intensive 
oversight  and  technical  assistance  at  a  level 
not  normally  accorded  these  types  of  programs, 
were  in  a  unique  position  to  influence  pro¬ 
gram  results.  At  several  locations,  this  as¬ 
sistance  greatly  improved  program  operations; 
at  others,  this  special  attention  salvaged 
marginal  operations.  These  actions  are  in¬ 
dicative  of  the  degree  of  oversight  necessary 
to  maintain  quality  youth  programs,  but  one 
can  only  speculate  as  to  the  fate  of  the  En¬ 
titlement  demonstration  without  the  oversight 
it  received.  (See  pp.  47  to  51.) 

RECOMMENDATIONS  TO  THE 
SECRETARY  OF  LABOR 


If  Entitlement  is  extended  or  expanded,  or 
a  similar  program  is  adopted,  GAO  recommends 
that  the  Secretary 

— require  sponsors  to  strictly  enfore  reason¬ 
able  attendance  and  academic  standards 
thereby  enhancing  the  program's  credi¬ 
bility  and  effectiveness  for  both  partici¬ 
pants  and  the  general  public; 

— increase  efforts  to  promote  cooperative 
relationships  between  sponsors,  schools, 
and  other  organizations  serving  youths; 
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“help  sponsors  upgrade  worksite  supervi¬ 
sion,  job  development,  participant  coun¬ 
seling,  and  monitoring,* 

— create,  implement,  and  enforce  adequate 
controls  to  prevent  substitution;  and 

— establish  and  enforce  requirements  that 
limit  program  management  costs.  (See 
p.  54.) 

CORPORATION  AND  SPONSOR  COMMENTS 

GAO  requested  comments  from  Labor,  the 
Corporation,  and  the  11  sponsors  included 
in  the  review.  The  Corporation  and  five 
sponsors  submitted  comments.  (See  pp.  66 
to  80.)  The  Corporation  and  the  Baltimore 
sponsor  disagreed  with  GAO’s  approach  to 
this  study.  GAO's  detailed  analysis  of 
Corporation  and  sponsor  concerns  starts 
on  page  54 .  After  the  30-day  comment  per¬ 
iod  provided  by  Public  Law  96-226  expired. 
Labor  submitted  comments  which  concurred 
with  all  of  GAO's  recommendations  to  the 
Secretary.  Labor's  comments,  which  are  in¬ 
cluded  as  appendix  X,  have  not  been  ana¬ 
lyzed  in  this  report. 
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YOUTH  UNEMPLOYMENT — A  PERSISTENT 
PROBLEM  THAT  HAS  RESISTED  SOLUTION 


Youth  unemployment  has  been  a  persistent  national  prob¬ 
lem.  In  1954  the  unemployment  rate  for  16-  to  24-year-olds 
was  10.6  percent  while  the  overall  rate  was  5.6  percent. 

Nearly  25  years  later,  at  the  end  of  1978,  the  rate  for  16- 
to  24-year-olds  was  at  12.2  percent  compared  to  a  6-percent 
overall  rate. 

Such  youth  unemployment  statistics,  however,  do  not  be¬ 
gin  to  illustrate  the  magnitude  of  the  problem.  Teenagers, 
for  example,  have  much  higher  unemployment  rates  than  older 
youths.  High  school  dropouts,  low-income  and  minority  (partic¬ 
ularly  black)  youths  also  have  higher  unemployment  rates  than 
other  youths.  While  the  official  teenage  unemployment  rate  in 
the  ghettos  is  40  percent,  the  Urban  League  suggests  that  the 
real  rate  is  60  percent.  Also,  as  pointed  out  in  congressional 
testimony  before  the  House  Committee  on  Education  and  Labor, 
unemployment  statistics  do  not  convey  the  reduced  sense  of 
self-worth  or  the  dashed  hopes  and  dreams  caused  by  unemploy¬ 
ment.  They  do  not  convey  the  pervasive  alienation  that  can 
occur  among  unemployed  youths  who  feel  they  cannot  make  a 
useful  contribution  to  society. 

The  seriousness  of  the  youth  unemployment  problem  is  a 
matter  of  controversy.  One  contention  is  that  it  is  not 
serious  because  youths  have  few  financial  responsibilities 
and,  therefore,  are  not  greatly  hurt  when  unemployed.  Another 
argument  is  that  many  unemployed  youths  are  students  looking 
for  part-time  jobs,  which  causes  unemployment  statistics  to 
overstate  the  true  extent  of  the  problem.  On  the  other  hand, 
some  contend  that  youth  unemployment  is  a  serious  problem 
which  translates  into  lower  earnings  later  in  life  due  to 
lost  job  experience.  Also,  youth  unemployment  may  result 
in  increased  crime  and  alienation  which  may  have  long-term 
effects . 

CONGRESSIONAL  CONCERN  ABOUT  YOUTH 
UNEMPLOYMENT  LEADS  TO  NEW  INITIATIVES 


In  the  early  1960s,  the  Congress  determined  that  youth 
unemployment  was  a  serious  problem  needing  attention.  As  a 
first  step,  the  Congress  provided  funds  for  youth  development 
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and  training  under  the  Manpower  Development  and  Training  Act 
of  1962.  In  1964,  the  Economic  Opportunity  Act  established 
two  major  programs  for  youths — the  Neighborhood  Youth  Corps 
and  the  Job  Corps.  With  the  passage  of  the  Comprehensive 
Employment  and  Training  Act  (CETA)  in  1973,  youths  continued 
to  participate  in  the  Job  Corps  and  an  offshoot  of  the  Neigh¬ 
borhood  Youth  Corps,  which  was  known  as  the  Summer  Program 
for  Economically  Disadvantaged  Youth.  In  addition,  youths 
participated  to  a  significant  extent  in  other  CETA  program 
activities,  such  as  work  experience  which  generally  operates 
on  a  year-round  basis. 

Based  on  a  concern  about  continued  high  unemployment  rates 
for  youths,  the  President  in  early  1977  requested  funds  for  new 
youth  employment  programs  as  part  of  the  job  creation  component 
of  his  economic  recovery  package.  The  resulting  legislation, 
the  Youth  Employment  and  Demonstration  Projects  Act  of  1977 
( YEDPA) ,  which  amended  CETA,  authorized  four  additional  youth 
programs . 

YEDPA  was  an  admission  that,  despite  years  of  experience 
and  the  expenditure  of  billions  of  dollars  for  youth  employment 
and  training  programs,  no  one  knew  how  to  solve  the  chronic 
youth  unemployment  problem.  YEDPA  is  an  attempt  to  test  var¬ 
ious  approaches  to  determine  what  works  best  so  that  effective 
youth  programs  can  be  developed.  The  following  programs  are 
YEDPA  components : 

1.  The  Young  Adult  Conservation  Corps — operated  jointly 
by  the  Departments  of  Labor,  Agriculture,  and 
Interior — provides  employment  for  out-of-school 
youths  16  through  23  years  of  age  on  conservation 
projects  on  public  lands. 

2 .  The  Youth  Community  Conservation  and  Improvement 
Projects  ( YCCIP ) — designed  to  employ  16-  to  19-year- 
old  out-of-school  youths  in  well-supervised  projects 
with  tangible  outputs  benefiting  the  community. 


3.  The  Youth  Employment  and  Training  Program  ( YETP ) — 
designed  to  provide  a  full  range  of  activities  and 
services,  including  job  experience,  training,  occu¬ 
pational  information,  and  supportive  services  for 
in-school  and  out-of-school  16-  to  21-year-olds. 

The  program  also  serves  14-  and  15-year-old 


in-school  youths.  YEDPA  authorizes  some  YETP  funds 
to  be  used  for  innovative  and  experimental  projects. 


4.  The  Youth  Incentive  Entitlement  Pilot  Projects 

(Entitlement)--designed  to  provide  jobs  and  test  the 
efficacy  and  impact  of  guaranteeing  jobs  for  all 
16-  to  19-year-old  disadvantaged  youths  residing 
in  a  designated  geographic  area  and  attending  school 
or  who  are  willing  to  return  to  school . 

More  than  $3  billion  has  been  provided  for  YEDPA  programs 
for  fiscal  years  1978-80.  Most  of  these  funds  have  been  dis¬ 
tributed  by  formula  to  all  460  CETA  prime  sponsors,  usually 
State  and  local  governments,  to  operate  YETP  and  YCCIP.  The 
Congress  has  provided  $222  million  for  the  Entitlement  experi¬ 
ment  which  is  limited  to  17  prime  sponsors.  For  these  prime 
sponsors.  Entitlement  funds  represent  the  vast  majority  of 
their  financial  resources  for  youth  programs. 

Entitlement,  one  of  the  largest  demonstration  programs 
the  Federal  Government  has  ever  attempted  in  the  field  of 
youth  employment,  is  unique  in  several  respects.  Any  youth 
who  meets  the  eligibility  requirements  is  guaranteed  a  job. 

It  is  the  Nation's  first  job  guarantee  experiment. 

Entitlement  is  also  unique  because  it  requires  a  commit¬ 
ment  to  an  educational  program  as  a  condition  of  employment. 
Program  participants  must  remain  in  or  return  to  school  and 
meet  established  job  and  academic  performance  standards. 

Youths  work  part-time  during  the  school  year  and  full-time 
during  the  summer.  To  successfully  complete  Entitlement,  a 
youth  should  obtain  either  a  high-school  diploma  or  its  equiv¬ 
alent.  Youths  who  perform  unsatisfactorily  in  school  or  at 
work  should  be  terminated  from  Entitlement.  Youths  should 
also  be  terminated  when  they  become  20  years  old  or  are  no 
longer  eligible  because  of  family  income  or  the  location  of 
their  residence. 

A  third  unique  aspect  of  Entitlement  is  that  it  gives 
wage  subsidies  to  private  employers  that  provide  jobs  for 
youths.  The  legislation  allowed  Labor  to  experiment  with 
a  variety  of  wage  subsidies  up  to  100  percent  to  encourage 
private  employers  to  participate.  In  addition  to  being  a 
demonstration,  it  is  a  large-scale  operating  program. 

Entitlement  is  typical  in  that  many  job  experiences  are 
identical  to  those  in  other  CETA  programs.  Other  than 
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private  sector  worksites  (which  account  for  roughly  20  percent 
of  participant  work  hours),  work  locations  and  the  type  of 
work  performed  by  program  participants  were  often  indistin¬ 
guishable  from  other  CETA  youth  program  worksites. 

We  reviewed  Entitlement  because  it  is  the  most  radical 
departure  from  traditional  youth  employment  programs.  En¬ 
titlement  started  in  March  1978  and  was  originally  scheduled 
to  terminate  in  June  1979.  The  program  was  later  extended 
through  September  1980.  The  President  has  submitted  a  youth 
legislative  proposal  to  the  Congress  which  would  incorporate 
the  concepts  of  YETP,  YCCIP,  and  Entitlement  in  a  new  youth 
initiative . 

Recent  GAO  reports  on  youth  employment  and  training 
programs  are  listed  in  appendix  I . 

PROGRAM  RESPONSIBILITY 

The  Department  of  Labor's  Office  of  Youth  Programs  within 
the  Employment  and  Training  Administration  has  overall  responsi¬ 
bility  for  supervising,  managing,  and  directing  Entitlement. 
However,  due  to  Labor's  strained  staff  capacity,  the  desire 
to  limit  Federal  workforce  expansion.  Entitlement's  short-term 
nature,  and  the  need  for  special  expertise  in  evaluation,  re¬ 
search,  and  demonstration.  Labor  awarded  a  contract  of  about 
$15  million  to  the  Manpower  Demonstration  Research  Corporation 
( MDRC )  of  New  York  City  to  manage  and  evaluate  Entitlement. 

MDRC  is  a  nonprofit,  publicly  supported,  private  organ¬ 
ization  which  designs,  manages,  and  evaluates  social  programs. 

It  was  established  in  1974  with  support  from  public  and  pri¬ 
vate  agencies  interested  in  pursuing  innovative  ways  to  treat 
some  of  society's  most  difficult  and  longstanding  problems. 

MDRC  is  governed  by  a  board  of  directors  and  has  a  staff  of 
more  than  100  persons. 

Under  Labor's  overall  direction,  MDRC's  Entitlement  re¬ 
sponsibilities  include  (1)  helping  Labor  negotiate  work  plans 
and  budgets  for  selected  prime  sponsors,  (2)  developing  and 
supervising  demonstration  procedures,  (3)  implementing  and 
supervising  the  research  design,  (4)  implementing  an  Entitle¬ 
ment  information  system,  (5)  analyzing  the  consistency  of 
operations  with  demonstration  and  research  requirements,  and 
(6)  identifying  training  and  technical  assistance  needs.  In 
addition,  MDRC  helped  select  Entitlement  prime  sponsors. 
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COMPETITIVE  SELECTION  PROCESS  USED 
TO  SELECT  BEST  PRIME  SPONSORS 


Because  of  the  expected  high  cost  of  a  guaranteed  job 
program,  Labor  limited  Entitlement  to  a  few  prime  sponsors. 
Labor  developed  a  national  competitive  selection  process 
consisting  of  a  preapplication  and  a  final  application  phase 
to  guarantee  the  selection  of  prime  sponsors  best  able  to 
meet  administrative  and  research  needs.  This  two-step  process 
included  site  visits  before  final  selection  so  that  the  deci¬ 
sion  would  be  based  on  more  than  grantsmanship  capabilities. 

The  competitive  process  was  designed  to  yield  a  wide  economic 
and  geographic  range  of  sponsors,  including  large  and  small 
cities,  high-density  urban  areas,  and  sparsely  populated  rural 
regions,  and  areas  of  varying  unemployment  rates,  school  drop¬ 
out  rates,  and  racial  and  ethnic  composition. 

Due  to  cost.  Labor  limited  Entitlement  to  a  few  large- 
scale  tests  and  additional  smaller  tests.  Since  YEDPA  sug¬ 
gested  several  secondary  experiments  and  innovations.  Labor 
designed  a  two-tier  approach.  Large-scale,  or  Tier  I,  projects 
would  test  the  feasibility  of  implementing  a  full-scale  job 
guarantee  program.  Each  Tier  I  sponsor  was  expected  to  provide 
between  3,500  and  10,000  jobs.  Because  the  Tier  I  demonstra¬ 
tions  were  expected  to  be  burdened  by  administrative  demands. 
Labor  allowed  smaller,  Tier  II,  projects  to  test  various  in¬ 
novative  approaches.  Tier  II  demonstrations  were  expected  to 
provide  between  200  and  1,000  jobs. 

All  CETA  prime  sponsors  were  allowed  to  submit  a  preap¬ 
plication  for  planning  funds  to  develop  a  final  application. 
Labor  requested  that  each  applicant  provide  specific  informa¬ 
tion  concerning:  demographic  characteristics  of  the  prime 
sponsor  area  and  the  proposed  Entitlement  area,  secondary 
school  systems  and  CETA  youth  programs,  proposals  for  organ¬ 
izational  and  administrative  arrangements,  and  a  project  budget 
and  estimates  of  job  demand  among  eligible  youths. 

Criteria  used  to  assess  preapplications  included  the 

— overall  quality  and  thoroughness  of  the  preapplication; 

— managerial,  administrative,  operating,  and  fiscal  capa¬ 
bility  of  the  prime  sponsor; 

— level  of  commitment  of  other  resources  to  the  Entitle¬ 
ment  project; 
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. — commitment  of  cooperation  and  participation  from  local 
school  systems,  labor  organizations,  and  other  local 
groups; 

— degree  to  which  Entitlement  would  be  integrated  with 
local  education,  career  development,  and  employment 
and  training  programs; 

— previous  experience  in  administering  related  programs; 

— proposed  cost  structure;  and 

— commitment  and  ability  to  provide  data  necessary  for 
research  and  analysis. 

Labor  established  an  Entitlement  selection  and  review 
panel  composed  of  10  individuals  from  academic,  labor, 
community-based  organization  (CBO),  youth  population,  and  man 
power  and  social  program  backgrounds.  All  panel  members  ex¬ 
cept  one,  a  college  student,  serve  on  the  MDRC  board  of  direc 
tors  . 

Labor  received  153  preapplications,  51  for  Tier  I  and 
102  for  Tier  II  projects.  Based  on  the  above  criteria,  Labor 
assessments,  and  the  geographical  and  socioeconomic  distribu¬ 
tion  mandated  by  YEDPA ,  the  panel  made  unanimous  recommenda¬ 
tions  to  Labor.  Labor  awarded  planning  grants  to  support  de¬ 
velopment  of  final  applications  to  18  and  16  prime  sponsors 
for  Tier  I  and  Tier  II  projects,  respectively. 

Labor  and  MDRC  review  teams  visited  each  prime  sponsor 
while  it  prepared  the  final  application.  The  visits  were 
designed,  among  other  things,  to  substantiate  preapplication 
information,  to  explain  program  requirements,  and  to  assess 
the  capabilities  and  commitment  of  key  operating  agencies. 

Labor  ranked  each  final  application  according  to  four 
criteria;  (1)  program  management  capability,  feasibility, 
and  commitment,  (2)  operational  feasibility,  (3)  management 
information  and  fiscal  systems,  and  (4)  budget  analysis  and 
local  resource  commitments.  A  fifth  criterion,  innovative 
features,  was  used  to  rate  Tier  II  proposals.  The  review 
panel  considered  all  available  materials  (preapplications, 
final  applications,  Labor  assessments,  field  visit  reports, 
and  ratings)  and  made  its  recommendations  to  Labor,  which 
selected  17  prime  sponsors  to  operate  the  Entitlement  demon¬ 
stration.  Labor  awarded  grants  to  7  Tier  I  and  10  Tier  II 
prime  sponsors.  (See  app.  II.) 


OBJECTIVES,  SCOPE,  AND  METHODOLOGY 


We  examined  the  following  eight  areas: 

— Admin istration. 

— Management  costs. 

— Recruitment,  enrollment,  and  assignment  to  jobs. 

— Maintenance  of  effort  and  substitution. 

— Cooperation  and  coordination  with  CBOs,  local  educa¬ 
tion  agencies,  the  private  sector,  and  other  agencies 
serving  youths. 

— Work  experience. 

— Academic  and  attendance  standards. 

— Monitoring  and  evaluation. 

Our  review  generally  covered  fiscal  years  1978,  1979,  and  part 
of  1980. 

We  made  our  review  at  Labor  headquarters  in  Washington, 
D.C.,  and  in  7  of  Labor's  10  regions:  Atlanta,  Boston,  Chicago, 
Dallas,  Denver,  Philadelphia,  and  Seattle.  We  visited  the  head¬ 
quarters  of  MDRC  in  New  York  City  and  Abt  Associates,  Inc.,  an 
MDRC  subcontractor  in  Cambridge,  Massachusetts. 

We  visited  the  7  Tier  I  prime  sponsors  and  4  of  the  10 
Tier  II  sponsors:  Albuquerque,  New  Mexico;  Hillsborough  County, 
New  Hampshire;  1/  Dayton,  Ohio;  and  Philadelphia,  Pennsylvania. 
We  also  visited  selected  prime  sponsor  grantees  and  contractors, 
including  local  education  agencies  and  CBOs . 

Using  standardized  questionnaires,  we  interviewed  848  par¬ 
ticipants  and  460  supervisors  at  350  Tier  I  worksites.  We  also 
visited  37  worksites  at  the  four  Tier  II  locations  and  inter¬ 
viewed  76  participants  and  59  supervisors.  In  most  cases. 


^/Entitlement  project  location  was  Nashua,  New  Hampshire. 
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these  visits  were  unannounced.  Interviews  at  Tier  I  and  II 
locations  took  place  in  two  phases,  a  summer  phase  when  the 
participants  were  working  full  time  and  a  school  year  phase 
when  the  participants  were  working  part  time.  Summer  inter¬ 
views  took  place  during  July  and  August  1979;  school  year  in¬ 
terviews,  between  September  and  December  1979. 

Sampling  methodology 

We  used  statistical  sampling  techniques  to  evaluate 
youths'  Entitlement  program  work  experiences.  On  the  basis 
of  a  statistical  sample  of  worksites  at  each  Tier  I  location, 
we  obtained  information  from  participants  as  well  as  their 
supervisors  and  evaluated  the  worksites.  This  approach  al¬ 
lowed  us  to  draw  conclusions  about  Tier  I  Entitlement  loca¬ 
tions  as  a  whole.  Because  Tier  II  sites  were  not  statis¬ 
tically  selected,  the  results  at  these  sites  are  not  project- 
able.  Tier  I  sites  accounted  for  90  percent  of  the  program 
participants  through  November  1979  and  92  percent  of  reported 
Entitlement  expenditures. 

Statistical  sampling  enabled  us  to  draw  conclusions 
about  the  universe  of  interest  (worksites)  on  the  basis  of 
information  on  a  sample  of  that  universe.  The  results  from 
a  statistical  sample  are  always  subject  to  some  uncertainty 
(i.e.,  sampling  error)  because  only  a  portion  of  the  universe 
has  been  selected  for  analysis.  Our  sample  size  was  large 
enough  to  keep  the  sampling  error  small.  The  sampling  error 
consists  of  two  parts:  confidence  level  and  range.  The  con¬ 
fidence  level  indicates  the  degree  of  confidence  that  can  be 
placed  in  estimates  derived  from  the  sample.  The  range  is 
the  upper  and  lower  limits  between  which  the  actual  universe 
value  may  be  found. 

For  example,  a  random  sample  of  worksites  showed  that 
33  percent  were  in  the  private,  for-profit  sector.  Using  the 
sampling  error  formula,  we  are  95-percent  confident  that  the 
true  percentage  of  private  worksites  in  the  universe  would  be 
within  plus  or  minus  5  percentage  points  of  the  sample  re¬ 
sults.  Thus,  if  all  worksites  in  the  universe  were  checked, 
the  chances  would  be  95  in  100  that  the  actual  percentage  of 
private,  for-profit  worksites  would  be  between  28  and  38  per¬ 
cent  . 

Using  a  worksite  as  our  sampling  unit,  we  devised  a  two- 
stage  stratified  sampling  plan.  From  about  4,700  Tier  I 
worksites  in  existence  in  June  1979,  we  selected  350  for 
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evaluation — 50  at  each  Tier  I  location.  This  sample  size  was 
determined  so  that  the  expected  sampling  error  on  a  program¬ 
wide  finding  would  not  exceed  plus  or  minus  5  percentage 
points  at  the  95-percent  confidence  level.  The  sampling 
error  on  a  finding  at  any  particular  Tier  I  location  was  ex¬ 
pected  to  be  about  8  percentage  points  at  the  95-percent 
confidence  level. 

To  select  the  sites,  we  stratified  all  worksites  at 
each  Tier  I  location  by  the  number  of  Entitlement  partici¬ 
pants  assigned  to  the  site.  Based  on  this  stratification, 
we  randomly  selected  50  sites  at  each  Tier  I  location  using 
random  number  tables.  At  each  site  selected,  we  interviewed 
all  participants  if  five  or  fewer  were  present.  At  worksites 
with  more  than  five  participants,  we  used  random  numbers  to 
select  five  from  those  present.  In  addition,  we  interviewed 
the  supervisors  of  all  selected  participants.  To  solicit 
responses  from  both  participants  and  supervisors,  we  used  a 
structured  interview  form  which  was  uniform  for  all  locations 
We  used  the  supervisors'  and  participants'  responses,  our 
observations,  and  a  review  of  records  as  our  basis  for  asses¬ 
sing  each  sampled  worksite. 

We  then  projected  the  results  of  these  worksite  assess¬ 
ments  to  what  we  estimate  to  be  5,200  worksite  locations. 

This  worksite  estimate  rose  from  4,700  due  to  the  fact  that 
some  worksites  had  subsites.  We  computed  these  projections 
using  weights  based  on  (1)  the  number  of  sites  sampled  in 
each  stratum  at  each  location  and  (2)  the  number  of  sites  in 
each  stratum  at  each  location.  The  estimates  derived  from 
the  sample  of  participants  and  projected  to  the  universe  were 
calculated  using  weights  based  on  the  number  of  (1)  partici¬ 
pants  interviewed  at  each  site,  (2)  participants  present  at 
each  site,  (3)  sites  sampled  in  each  stratum  at  each  location 
and  (4)  sites  in  each  stratum  at  each  location. 

We  estimate  that  the  participant  responses  can  be  pro¬ 
jected  to  a  universe  of  about  14,800  Tier  I  participants. 

The  actual  number  of  participants  assigned  is  probably  more 
since  our  projections  are  based  on  participants  present  on 
the  days  of  our  site  visits.  In  fact,  we  estimate  that 
there  were  about  20,000  participants  assigned  to  the  5,200 
sites . 


CHAPTER  2 


MOST  WORKSITES  PROVIDED  YOUTHS  WITH  ADEQUATE 
WORK  EXPERIENCES  WHILE  OTHERS  NEED  IMPROVEMENT 

The  Congress  specifically  mandated  that  YEDPA  programs, 
including  Entitlement,  offer  participants  meaningful  opportu¬ 
nities  to  earn  and  learn.  To  carry  out  this  mandate.  Labor, 
in  its  Planning  Charter  for  YEDPA,  emphasized  the  goal  of  a 
quality  youth  work  experience. 

Based  on  worksite  visits,  we  estimate  that  of  all  Tier  I 
worksites  in  the  program 

— 81  percent  provided  environments  where  youths  were 

encouraged  to  develop  and  demonstrate  good  work  habits, 

— 73  percent  provided  youths  with  enough  useful  work, 
and 

— 74  percent  provided  youths  with  adequate  supervision. 

Although  not  statistically  projectable,  the  work  experience 
characteristics  of  the  Tier  II  sites  we  visited  were  about  the 
same  as  the  Tier  I  sites. 

Most  worksites  provided  all  three  elements  crucial  to  an 
adequate  work  experience;  however,  many  others  did  not.  We 
estimate  that  about  3,000  (58  percent)  of  the  5,200  worksites 
met  our  criteria  for  all  three  elements,  while  the  remaining 
2,200  (42  percent)  lacked  one  or  more  of  these  elements: 

1,100  (21  percent)  had  serious  problems — failing  to  meet  our 
criteria  for  two  or  more  elements — and  another  1,100  (21  per¬ 
cent)  had  moderate  problems — failing  to  meet  our  criteria  for 
one  element.  We  estimate  that  50  percent  of  all  Tier  I  par¬ 
ticipants  were  assigned  to  those  worksites  (42  percent)  which 
had  one  or  more  problems . 

To  evaluate  worksite  adequacy  in  any  of  the  three  cate¬ 
gories,  we  considered  supervisor  and  participant  responses 
and  our  observations  of  activities  at  the  site.  We  also 
analyzed  data  from  our  site  visits  to  identify  factors  that 
were  characteristic  of  adequate  and  inadequate  worksites. 

Although  most  worksites  provided  adequate  work  experi¬ 
ences,  others  need  improvement.  In  varying  degrees,  sponsors 
did  not: 
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— Assure  that  youths  received  adequate  supervision. 

— Develop  and  sustain  a  sufficient  quantity  of  jobs  in 
the  right  locations. 

— Adequately  counsel  youths. 

— Provide  effective  monitoring  of  worksites.  Problems  of 
poor  supervision,  marginal  work  experiences,  and  time 
and  attendance  or  payroll  violations  that  should  have 
been  obvious  often  went  undetected  or,  if  detected, 
uncorrected . 

RESULTS  OF  WORKSITE  ASSESSMENTS 

Labor's  Entitlement  regulations  stress  that  worksites 
were  to  have  capable  supervision  and  provide  structured,  pro¬ 
ductive  work  settings.  The  jobs  were  to  introduce  youths  to 
the  habits  of  successful  work  life  and  entry  level  or  prep¬ 
aratory  skills.  Additional  requirements  included  develop¬ 
ing  enough  jobs  to  minimize  the  time  between  enrollment  and 
assignment  to  a  worksite  and  providing  monitorable  attendance 
and  productivity  standards. 

Work  habits 


Good  work  habit  development  is  a  critical  element  in  any 
youth  employment  program.  For  many  participants  an  Entitle¬ 
ment  job  is  their  first  work  experience.  Based  on  our  inter¬ 
views  with  participants,  we  estimate  that  81  percent  of  the 
Entitlement  participants  never  had  a  prior  job.  Consequently, 
it  is  extremely  important  that  these  youths  be  exposed  to 
and  guided  in  the  development  of  good  work  habits. 

On  the  basis  of  worksite  interviews  with  participants  and 
their  supervisors  as  well  as  our  recorded  observations,  we 
evaluated  whether  the  worksite  provided  an  environment  where 
good  work  habits  were  encouraged.  In  making  our  evaluation, 
we  considered  factors,  such  as: 

— Whether  youths  came  to  work  regularly. 

— Whether  youths  came  to  work  on  time. 

— How  adequately  work  and  progress  of  participants  was 
evaluated . 
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— How  adequately  services,  other  than  job  training, 
were  given  to  participants. 

We  estimate  that  81  percent  of  all  worksites  provided  environ¬ 
ments  where  good  work  habits  were  encouraged .  Seventy-three 
percent  of  the  program  participants  were  at  these  sites. 

We  also  examined  our  data  to  identify  factors  that  were 
characteristic  of  sites  that  provided  good  work  habit  develop¬ 
ment.  According  to  our  analysis,  the  adequacy  of  supervision 
and  the  quantity  of  work  were  the  two  key  factors.  We  deter¬ 
mined  that  youths  at  sites  with  adequate  supervision  and  a 
sufficient  quantity  of  work  are  more  likely  to  exhibit  good 
work  habits.  Conversely,  youths  at  sites  with  inadequate 
supervision  or  not  enough  work  to  keep  them  busy  are  not 
likely  to  exhibit  good  work  habits.  Other  factors  were  con¬ 
sidered  but  did  not  prove  to  be  important  in  distinguishing 
between  sites  that  provided  or  did  not  provide  good  work  habit 
development.  These  factors  included  the  number  of  youths 
assigned  to  the  worksite,  the  number  of  youths  present  on 
the  day  of  our  visit,  whether  the  site  provided  full-time 
jobs  in  the  summer  or  part-time  jobs  during  the  school  year, 
and  whether  the  site  was  private,  for-profit  or  not. 

As  discussed  above  most  Entitlement  worksites  provided 
environments  where  youths  were  exposed  to  real  work  situa¬ 
tions  and  encouraged  to  develop  good  work  habits.  Some 
examples  follow: 

— A  Federal  agency  in  Cincinnati  employed  three  youths 
as  general  office  workers  whose  duties  included  filing, 
typing,  and  running  office  machines.  The  supervisor 
encourages  good  work  habits  by  treating  the  partici¬ 
pants  as  regular  employees.  The  participants  are 
supervised  at  all  times,  have  enough  work,  and  receive 
training  in  general  office  procedures. 

— A  supervisor  at  a  Boston  shoe  manufacturer  provides 
constant  direction  to  two  participants  in  the  account¬ 
ing  department.  The  participants  perform  the  same 
activities  as  regular  employees  and  are  assigned 
additional  work  as  they  complete  their  tasks. 

— A  supervisor  in  a  Cincinnati  retail  store  encourages 
good  work  habits  by  assuring  that  the  one  participant 
is  gainfully  employed  and  adheres  to  work-hour 
standards.  The  participant's  main  duties  include 
stocking  shelves,  unpacking  new  stock,  and  cleaning 
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the  sales  area.  In  addition,  the  supervisor  is  pro¬ 
viding  the  participant  with  sales  training. 


Other  sites,  however,  did  little  to  encourage  good  work 
habits.  For  example: 

— A  large  Detroit  high  school  employed  23  students  in 
custodial  work.  The  supervisor  routinely  allowed 
students  to  leave  after  they  finished  2  hours  of  work, 
although  they  were  paid  for  4  hours.  The  site  did 
not  have  enough  work  for  the  youths  assigned. 

— At  a  Boston  youth  activities  center,  four  participants 
did  no  work  during  the  3  hours  we  were  there.  One 
participant  read  a  magazine.  Another  was  sent  out  to 
get  lunch  for  the  other  employees.  In  addition,  two 
participants  who  reported  over  an  hour  late  for  work 
were  signed  in  by  their  supervisor  at  their  normal 
starting  times  and  paid  for  the  hours  not  worked. 

— In  Seattle,  one  youth  was  assigned  to  an  Armed  Forces 
recruiting  office  as  a  receptionist  and  clerk  typist. 
In  violation  of  the  rules,  the  supervisor  allowed  the 
youth  to  call  in  sick  on  several  occasions  and  au¬ 
thorized  payment  for  these  absences.  The  supervisor 
permitted  the  youth  to  use  a  telephone  answering 
machine,  so  she  could  go  shopping  and  take  extended 
lunch  hours.  After  our  visit,  the  sponsor  removed 
the  youth  from  the  site  and  will  not  use  the  site 
until  a  quality  supervisor  is  assigned  there. 

Work  content 


In  evaluating  worksites  for  the  usefulness/meaningful¬ 
ness  of  jobs  provided  youths,  we  assessed  whether  the  jobs: 

(1)  were  useful  to  the  community  or  employer  and  (2)  provided 
youths  an  opportunity  to  learn  a  new  skill  or  improve  exist¬ 
ing  skills.  We  classified  worksites  as  providing  youths  with 
useful/meaningful  work  if  at  least  60  percent  of  youths'  daily 
activities  at  the  sites  met  either  of  these  two  criteria. 

Overall,  we  estimate  that  about  73  percent  of  the  work¬ 
sites  provided  youths  with  useful/meaningful  work.  Further, 
we  estimate  that  60  percent  of  all  Tier  I  participants  were 
at  these  sites.  Worksites  were  more  successful  in  providing 
youths  with  useful  jobs  than  they  were  in  providing  youths 
with  jobs  where  they  could  develop  new  or  use  existing 
skills  as  shown  below: 


Estimated  Percentage 

percentage  of  of  youths 
worksites  at  sites 


Useful  to  employer 

or  community  74  62 

Useful  to  participant  66  56 


We  also  examined  our  data  to  identify  factors  that  were 
characteristic  of  sites  that  provided  a  useful/meaningful  work 
experience.  According  to  our  analysis,  there  were  four  key 
factors,  of  which  quantity  of  work  was  most  important. 
Supervision,  minimizing  idle  time,  and  the  number  of  youths 
assigned  to  a  site  were  also  important.  Other  factors  con¬ 
sidered  which  did  not  prove  to  be  important  included  the 
number  of  youths  present  at  the  worksite,  whether  the  site 
provided  full-time  jobs  in  the  summer  or  part-time  jobs 
during  the  school  year,  and  whether  the  site  was  private, 
for-profit  or  not. 


Our  analysis  indicates  that  a  useful/meaningful  work 
experience  is  not  likely  to  exist  at  worksites  where  too  many 
participants  are  assigned,  where  there  is  an  insufficient 
quantity  of  work,  and  where  supervision  is  inadequate. 
Following  are  some  representative  examples  of  worksites  we 
evaluated  as  adequate. 


— A  Detroit  hospital  gave  29  students  an  excellent  job 
experience  and  an  opportunity  to  develop  occupational 
skills  and  good  work  habits.  Twenty-six  of  the  stu¬ 
dents  worked  as  nurses  aides  in  various  departments 
and  three  performed  clerical  duties.  The  site  super¬ 
visor  emphasized  good  work  habits,  enforced  work- 
hour  rules,  and  motivated  the  youths  to  accept  respon¬ 
sibility.  In  return,  the  youths  were  providing  a 
valuable  service. 


— Again  in  Detroit,  a  cultural  center  used  six  partici¬ 
pants  as  program  aides.  They  assisted  in  planning 
activities,  aided  students  in  various  workshops,  and 
gave  tours  of  the  site.  The  participants  were  actively 
involved  in  the  operations  of  the  site  and  had  daily 
contact  with  the  public  and  professionals  in  the 
areas  of  art,  music,  and  crafts.  In  addition  to  pro¬ 
viding  a  useful  service,  some  of  the  participants  were 
interested  in  continuing  to  work  in  these  areas  and 
were  receiving  good  training. 


Other  sites  did  not  provide  a  sufficient  quantity  of 
useful/meaningful  work.  For  example: 

— In  Baltimore,  five  recreation  sites  employed  45  youths 
as  recreation  aides.  Idleness  was  a  problem  at  these 
sites,  and  the  youths,  when  not  obviously  idle,  parti¬ 
cipated  in  sports  activities  or  watched  other  youths 
engaged  in  recreational  activities.  The  lack  of 
enough  work  and  adequate  supervision  prevented  youths 
at  most  recreation  sites  from  even  receiving  training 
in  rudimentary  work  habits. 

— The  City  of  Detroit  transportation  department  employed 
22  youths  to  fill  requisitions  and  take  inventory. 

Some  youths  worked  during  the  day  when  they  were  super¬ 
vised  by  regular  employees  and  were  given  a  sufficient 
quantity  of  work  to  do.  Several  youths,  however, 
worked  in  the  evening  when  there  was  an  insufficient 
quantity  of  work  and  not  enough  supervision.  These 
youths  sat  around,  and  some  left  work  early — with  pay. 
The  supervisor  believed  he  could  only  supervise  and 
keep  busy  a  maximum  of  12  participants. 

In  substance,  most  Entitlement  jobs  were  similar  to 
those  in  other  youth  programs,  with  clerical,  custodial,  and 
recreational  jobs  predominating.  They  tend  not  to  be  skill 
oriented  in  the  sense  that  classroom  or  on-the-job  training 
are  prerequisites  for  performing  the  job  tasks.  We  estimate, 
however,  that  about  50  percent  of  the  participants  received 
on-the-job  training,  even  though  few  youths  entered  the  pro¬ 
gram  with  training  as  an  objective.  In  contrast,  only  about 
6  percent  received  any  formal  classroom  training. 

THE  QUALITY  OF  YOUTHS'  JOB 
EXPERIENCES  CAN  BE  IMPROVED 


Although  the  majority  of  Entitlement  worksites  provided 
an  adequate  work  experience,  improvements  in  several  program 
elements  are  needed  to  enhance  the  quality  of  the  youths'  job 
experience.  Supervision,  job  development,  counseling,  and 
worksite  monitoring  are  all  important  elements  in  providing 
quality  job  experiences,  with  supervision  the  most  important 
element . 


15 


In  analyzing  the  factors  contributing  to  the  quality  of 
the  work  experience,  supervision  was  a  significant  factor 
in  work  habit  development  and  meaningful  work  content. 

In  evaluating  supervision,  we  assessed  how  adequately 
supervisors 

— understood  program  object- ives; 

— enforced  work-hour  standards; 

— controlled  participant  behavior; 

— minimized  idle  time; 

— impressed  upon  participants  the  need  for  good  work 
habits ; 

— assigned,  trained,  and/or  guided  participants  in  their 
job  tasks;  and 

— evaluated  participant  work  and  progress. 

We  obtained  information  on  thes  »  issues  by  interviewing 
worksite  supervisors  and  part ic i oants ,  observing  worksite 
conditions,  and  examining  records. 

We  estimate  that  74  percent  of  the  worksites  provided 
youths  with  adequate  supervision.  These  sites  had  67  per¬ 
cent  of  all  Tier  I  participants . 

We  also  examined  our  data  to  identify  factors  that  were 
characteristic  of  sites  that  had  adequate  supervision.  We 
analyzed  the  following  factors: 

— Number  of  youths  assigned  to  the  worksite. 

— Number  of  youths  present  on  the  day  of  our  visit. 

— Whether  enough  work  was  available. 

— Whether  the  site  provided  full-time  jobs  in  the 
summer  or  part-time  jobs  during  the  school  year. 

— Whether  the  site  was  private,  for-profit  or  not. 


— Whether  the  site  was  in  an  urban  or  rural  area. 


— Whether  attendance  was  recorded. 

— Whether  attendance  was  taken  in  a  timely  manner. 

— Whether  attendance  records  were  complete. 

According  to  our  analysis,  only  three  of  these  factors 
were  significant  in  separating  sites  with  adequate  supervision 
from  those  without.  In  order  of  importance,  these  factors 
were;  whether  enough  work  \.as  available  to  keep  the  partici¬ 
pants  busy,  whether  attendance  records  were  complete,  and 
whether  attendance  was  recoided  at  all.  Once  the  effect  of 
these  factors  was  considered,  none  of  the  others  was  signifi¬ 
cant.  Examples  of  sites  exhibiting  significant  characteris¬ 
tics  of  adequate  supervision  included  the  following; 

— Six  participants  at  a  Boston  high  school  worked  on  a 
general  maintenance  <  rew.  They  worked  side-by-side 
with  the  regular  stair  preparing  the  school  for 
occupancy  in  the  fall.  Their  job  tasks  included 
washing  the  windows  and  floors,  landscaping,  and 
painting.  The  supervisor  had  a  schedule  of  projects 
to  be  completed  before  school  began.  The  supervisor 
enforced  work-hour  standards,  minimized  idle  time, 
and  encouraged  good  work  habit  development. 

— At  a  Nashua  recreation  facility,  four  participants 
were  engaged  in  different  jobs.  Two  youths  assisted 
their  supervisor  in  preparing  and  serving  food.  The 
supervisor  scheduled  their  activities  allowing  for 
little  idle  time.  Another  participant  worked  in  a 
clerical  position  answering  the  phone,  typing,  pre¬ 
paring  the  payroll,  and  enrolling  new  club  members. 

The  fourth  participant  was  a  recreation  aide  and 
distributed  equipment  and  monitored  various  youth 
activities . 

— At  a  Cincinnati  special  education  center,  one  partici¬ 
pant  was  learning  to  type,  operate  office  machines, 
and  prepare  educational  material.  The  supervisor 
maintained  constant  supervision  over  the  participant 
and  assured  that  she  was  kept  busy. 


Where  too  many  participants  were  assigned  for  the  amount 
of  work  available  and  work-hour  standards  were  not  enforced, 
supervision  was  generally  inadequate.  For  example: 

— A  Mississippi  high  school  worksite  had  over  100  par¬ 
ticipants  assigned  primarily  to  custodial  functions. 
Some  worked  as  teachers  aides.  The  Vice  Principal 
readily  admitted  that  the  crew  was  too  large;  however, 
he  tolerated  the  situation  because  the  participants 
needed  the  money.  We  observed  a  great  deal  of  milling 
around,  little  useful  work,  and  no  intervention  on  the 
part  of  the  supervisors  to  at  least  minimize  the  chao¬ 
tic  conditions  at  this  site.  For  example,  we  observed 
one  youth  vacuuming  a  few  square  feet  of  floor  for 
about  30  minutes  while  talking  to  his  friends. 

— A  large  Cincinnati  worksite  employing  over  100  parti¬ 
cipants  divided  the  youths  into  groups  to  clean  vacant 
lots  throughout  the  city.  The  supervisor  of  one  group 
told  us  that  participants  are  frequently  absent,  and 
she  never  knows  how  many  youths  will  appear  each  day. 
Further,  no  action  was  taken  to  discipline  youths  who 
do  not  work  or  to  encourage  better  attendance. 

— Fifty-two  participants  at  a  Baltimore  recreation  center 
were  doing  no  work.  They  either  were  playing  games 
or  were  simply  idle.  Five  participants  took  a  trip 
to  an  out-of-State  amusement  park.  One  youth  skipped 
a  day's  work  because  he  was  to  attend  a  professional 
baseball  game  the  next  night  as  a  paid  activity. 
According  to  the  worksite  agreement,  the  youths  were 
supposed  to  be  working  in  maintenance,  clerical,  and 
recreational  positions.  Further,  youths  were  paid  for 
lunch  periods  and  there  were  numerous  time  and  attend¬ 
ance  discrepancies.  These  activities  occurred  with 
the  supervisor's  approval  and  no  attempt  was  made  to 
put  the  youths  to  work.  One  supervisor  was  fired  and 
three  others  were  suspended  after  we  brought  this 
situation  to  the  attention  of  program  officials  and 
they  had  confirmed  our  findings. 

We  estimate  that  supervisors  did  not  properly  prepare 
or  complete  attendance  records  at  32  percent  of  the  worksites. 
We  also  estimate  that  at  least  one  youth  was  absent  at  about 
22  percent  of  the  worksites  on  the  day  of  our  visits,  and  at 
26  percent  of  these  sites  at  least  one  youth  was  paid  for  the 
absence.  This  tended  to  be  a  problem  at  the  larger  sites. 
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In  some  cases,  improper  payments  were  caused  by  inade¬ 
quate  time  and  attendance  and  payroll  controls.  In  other 
cases,  supervisors  allowed  participants  to  presign  timesheets 
or  fill  them  in  at  the  end  of  a  pay  period  instead  of  at  the 
end  of  each  day.  Others  authorized  payment  because  they 
viewed  Entitlement  as  simply  an  income  transfer  program. 

Some  examples  of  time  and  attendance  problems  follow: 

— In  Boston,  we  reviewed  the  timecards  of  the  17  parti¬ 
cipants  absent  on  the  day  of  our  visit.  A  sponsor 
official  told  us  that  absenteeism  was  a  problem  and 
estimated  that,  on  any  given  day,  20  percent  of  all 
participants  failed  to  report  to  work.  Of  more  sig¬ 
nificance,  however,  was  the  fact  that  7  of  the 
17  participants  (41  percent)  were  paid  for  the  day 
they  were  absent. 

— Baltimore  had  similar  problems.  At  seven  sites  we 
visited,  15  (47  percent)  of  32  youths  absent  on  the 
day  of  our  visit  were  paid.  Baltimore  worksites 
also  had  significant  time  and  attendance  record¬ 
keeping  problems.  (See  p.  18.) 

The  worksite  supervisor  clearly  has  prime  responsibility 
for  enforcing  time  and  attendance  requirements.  We  believe 
that  improvements  in  this  area  would  lead  directly  to  better 
participant  work  habits. 


Job  development 

To  fulfill  the  Entitlement  job  guarantee.  Labor  regula¬ 
tions  required  sponsors  to  develop  jobs  which  were  easily 
accessible  to  participants  and  reasonably  close  to  their 
homes.  Further,  there  had  to  be  enough  jobs  so  that  the 
waiting  time  between  enrollment  and  job  assignment  was 
minimized.  Generally,  sponsors  relied  on  the  traditional 
providers  of  jobs  from  the  summer  youth  program;  e.g.,  city 
and  State  agencies,  CBOs,  and  schools.  Initially,  they 
appeared  to  have  developed  a  sufficient  quantity  of  jobs. 
However,  most  Tier  I  sponsors  had  difficulty  creating  or 
maintaining  an  adequate  number  of  part-time  jobs  during  the 
school  year.  In  addition  to  problems  in  developing  jobs, 
some  sponsors  had  administrative  problems  in  the  timely 
assignment  of  youths  to  available  jobs. 


Mississippi  found  it  particularly  difficult  to  maintain 
enough  jobs.  Ironically,  it  had  done  a  fairly  good  job  in 
assigning  youths  at  the  start  of  the  program.  However,  when 
its  program  shifted  from  full-time  summer  jobs  to  part-time 
school  year  jobs  in  both  1978  and  1979,  a  severe  deficit  in 
the  quantity  of  accessible  jobs  occurred.  As  late  as  Decem¬ 
ber  1979,  about  18  percent  of  the  enrollees  were  in  a  hold 
status  pending  assignment  or  reassignment  to  a  job.  The 
lack  of  transportation,  school/work  scheduling  problems,  and 
coordination  difficulties,  mainly  between  the  sponsor  and 
its  major  subcontractor,  have  contributed  to  the  problem. 

Other  sponsors  also  had  difficulties  in  developing 
accessible  part-time  jobs  that  met  Entitlement's  10-hour  per 
week  minimum  work  requirement  during  the  school  year.  The 
location  of  available  jobs,  school  scheduling  problems,  and 
transportation  difficulties  affected  the  sponsors'  ability 
to  fulfill  Entitlement's  job  guarantee. 

To  resolve  some  of  these  problems,  Detroit  made  liberal 
use  of  fast  food  establishments  because  they  provide  flexible 
hours.  School  sites  were  also  a  major  source  for  placements; 
however,  many  were  overloaded.  The  sponsor’s  program  manaqer 
told  us  that  he  considers  school  sites  as  a  last  resort 
because  they  provide  low  quality  work  experiences.  In 
February  1980,  the  sponsor  placed  a  moratorium  on  further 
assignments  to  them  due  to  the  overloading. 

In  response  to  a  critical  job  shortage,  Cincinnati  cur¬ 
tailed  its  active  outreach  efforts  in  February  1980.  A 
sponsor  official  stated  that  by  1980  the  public  sector  was 
saturated,  but  they  were  attempting  to  place  additional  youths 
at  existing  public  and  private  worksites. 

At  the  inception  of  the  program,  the  many  participants 
who  claimed  their  guaranteed  job  overwhelmed  the  assignment 
capabilities  of  most  Tier  I  sponsors.  Some  participants 
waited  months  for  jobs  as  sponsors'  assignment  processes 
faltered.  Most  Tier  II  sponsors  found  it  easier  to  assign 
participants  to  jobs  because  of  their  smaller  numbers  of  par¬ 
ticipants.  As  the  program  progressed,  most  sponsors  reduced 
the  time  between  enrollment  and  assignment.  In  a  March  1980 
report,  MDRC  attributed  much  of  the  improvement  in  assignment 
processing  times  to  better  administrative  procedures,  partic¬ 
ularly  in  Boston  and  Detroit.  Of  some  significance,  however, 
is  the  fact  that  sponsors  had  far  fewer  youths  entering  the 
program  than  they  did  at  the  beginning  of  the  program. 
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Further,  many  youths  were  on  hold  and  not  working  pending 
initial  assignment,  reassignment,  or  termination.  MDRC  re¬ 
ported  that  as  of  December  1979,  about  2,200  youths  were  on 
hold  pending  a  first  assignment,  59  percent  of  whom  had  waited 
more  than  30  days.  Another  1,800  youths  were  on  hold  pending 
reassignment  or  termination.  Eighty-seven  percent  of  these 
youths  had  been  in  this  status  longer  than  30  days. 

Counseling 

According  to  their  grant  applications,  all  Tier  I  spon¬ 
sors  planned  to  provide  varying  degrees  of  counseling  to 
participants  on  such  matters  as  education,  health,  career 
or  job  placement,  and  work  habits.  Some  sponsors  planned  to 
furnish  these  services  directly,  while  others  planned  to  con¬ 
tract  with  other  agencies  for  some  services. 

Although  all  participants  might  not  have  required  a  full 
range  of  counseling  services,  many  needed  school  performance 
and  work  habit  counseling.  In  general,  many  Tier  I  sponsors 
placed  a  low  priority  on  counseling  compared  to  other  program 
components.  However,  more  youths  were  terminated  from  En¬ 
titlement  for  negative  reasons  than  positive  ones.  We  believe 
this  high  percentage  of  negative  terminations  may  indicate  a 
need  for  more  substantive  counseling  efforts. 

Through  December  1979,  an  MDRC  breakdown  of  the  34,000 
terminations  shows  the  following: 


Percent 

Positive: 


High  school  graduates 

26.7 

Negative: 

Dropped  out  of  school 

5.8 

Unsatisfactory  school 

performance 

3.9 

Unsatisfactory  job  performance 

18.2 

Lost  contact 

6.2 

34.1 

Neutral : 

No  longer  eligible 

10.0 

Left  for  other  program  or  job 

11.7 

Other 

17.5 

39.2 

100.0 
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At  most  Tier  I  locations,  "counselors"  spent  little 
time  counseling  youths.  They  were  primarily  involved  in  the 
payroll  process — picking  up  timesheets  and  delivering  checks. 
Further,  many  files  of  participants  we  interviewed  in  the 
summer  1/  lacked  adequate  documentation  on  counseling  activi¬ 
ties.  About  45  percent  of  these  files  had  no  evidence  of  any 
counseling  activity.  Thus,  in  many  cases  it  is  impossible  to 
determine  whether  active  or  terminated  participants  were  ever 
counseled . 

Of  the  files  containing  evidence  of  counseling,  about 
80  percent  indicated  that  participants  received  some  counsel¬ 
ing  on  work  habits.  Only  about  40  percent  contained  documen¬ 
tation  on  school  performance  counseling.  In  our  interviews 
with  participants,  less  than  half  stated  that  they  had  dis¬ 
cussed  their  future  after  graduation  with  their  counselor. 

Some  sponsors  did  not  develop  a  plan  to  help  participants 
graduating  from  high  school  obtain  unsubsidized  employment. 

In  this  regard,  Baltimore  and  Boston,  when  faced  with  a  large 
number  of  terminating  seniors,  maintained  the  participants 
in  their  Entitlement  jobs  by  funding  them  under  another  CETA 
program,  such  as  public  service  employment. 

Participants  at  some  of  the  Tier  II  sponsors  we  visited, 
particularly  Albuquerque  and  Nashua,  received  more  frequent 
and  in-depth  counseling.  We  believe  that  the  smaller  size  of 
these  programs  aided  these  efforts. 

Monitoring 

Labor *8  evaluations  of  other  youth  programs  showed  that 
participants  benefit  most  from  structured  and  demanding  work¬ 
site  activities.  Labor  also  found  that  effective  monitoring 
dramatically  improved  worksite  quality,  particularly  when 
sponsors  measured  whether  participants  were  actively  employed, 
productive,  and  supervised  and  met  attendance  and  performance 
standards.  Consequently,  Entitlement  guidelines  emphasized 
that  sponsors  not  only  provide  supervised  and  disciplined 
work  experiences  but  also  monitor  worksite  quality. 


1/We  did  not  review  counseling  files  in  the  fall. 


Initially,  most  sponsors  did  not  implement  effective 
monitoring  systems.  Although  Entitlement  started  in  the 
spring  of  1978,  sponsors  did  not  begin  to  seriously  monitor 
worksites  until  the  spring  and  summer  of  1979.  Cincinnati 
and  Detroit  did  no  worksite  monitoring  during  the  first 
year  of  the  program.  Cincinnati  monitored  many  of  its  work¬ 
sites  over  a  period  of  about  1  month  in  1979  in  preparation 
for  our  review. 

Some  sponsors  lacked  effective  monitoring  instruments 
and  used  untrained  personnel  as  monitors.  This  resulted  in 
some  Tier  I  sponsors  performing  little  more  than  limited 
compliance  reviews  of  worksite  time  and  attendance  data  and 
payroll  problems.  Baltimore,  Boston,  Cincinnati,  and  Miss¬ 
issippi  1/  used  checkoff  lists  to  record  supervisor  and  par¬ 
ticipant  responses,  but  monitors  commented  infrequently  or 
superficially  on  the  quality  of  worksites.  Baltimore  and 
Denver  used  inexperienced  CETA  public  service  employees  as 
monitors  without  giving  them  any  specific  training. 

Increased  sponsor  monitoring  during  and  after  the  summer 
of  1979  did  not  necessarily  lead  to  the  resolution  of  some 
serious  problems.  Problems  of  inadequate  supervision,  in¬ 
sufficient  quantity  of  work,  and  time  and  attendance  infrac¬ 
tions  that  should  have  been  obvious  went  undetected,  or,  if 
detected,  uncorrected  until  our  review.  For  example,  as 
previously  discussed,  Baltimore  and  Boston  had  numerous 
time  and  attendance  and  payroll  infractions.  Nearly  all  of 
the  50  sites  we  visited  in  Baltimore  had  one  or  more  of  the 
following  timekeeping  or  payroll  deficiencies: 

— Paid  absences,  latenesses,  and  meal  breaks. 

— Participants  maintaining  or  possessing  their  own 
timecards . 

— Timecards  signed  in  advance  or  at  the  end  of  a  time 
period  instead  of  daily. 

— Signatures  without  hours  shown  or  vice-versa. 

— Different  signatures  for  the  same  name. 


_1/After  the  completion  of  our  fieldwork,  Mississippi  told  us 
that,  in  addition  to  the  checklists  used  by  its  program 
monitors,  an  independent  monitoring  unit  prepares  narrative 
reports  on  worksite  conditions. 
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— Timecards  not  showing  actual  days  or  hours  worked. 

— Incorrect  totals  on  timecards. 

— Unapproved  changes  on  timecards. 

— No  sign-in  sheet,  unsupervised  sign-ins,  or  sign-in 
sheets  not  showing  actual  hours  worked. 

— Sign-in  sheets  inconsistent  with  timecards. 

Boston  had  many  of  the  same  problems.  Both  sponsors  took 
corrective  action  when  we  brought  these  problems  to  their 
attention,  and  in  late  1979,  both  revised  their  monitoring 
procedures.  The  changes  in  their  monitoring  procedures 
occurred  in  the  latter  stages  of  our  review;  consequently, 
we  are  unable  to  comment  on  their  effectiveness. 

In  some  cases,  problems  at  worksites  may  have  gone  un¬ 
detected  simply  because  of  the  infrequency  of  sponsors'  moni¬ 
toring  visits.  Because  of  program  management  cost  ceilings 
discussed  in  chapter  3,  Cincinnati,  Denver,  and  Seattle  did 
not  adequately  staff  monitoring  units.  Of  the  50  worksites 
we  visited,  Detroit  monitors  had  never  visited  18  and  had 
visited  13  others  just  once.  Monitoring  reports  are  super¬ 
ficial,  and  for  some  of  the  same  sites  we  visited,  the  monitors 
either  did  not  detect  problems  that  should  have  been  obvious 
or  failed  to  record  them  in  their  reports.  In  Boston,  26  of 
the  50  worksites  we  visited  failed  to  meet  at  least  one  of 
the  three  elements  crucial  to  an  adequate  work  experience. 
Sponsor  monitors  had  visited  11  of  these  26  sites  but  identi¬ 
fied  problems  at  only  4.  The  sponsor  had  taken  no  action 
to  correct  even  the  problems  raised  by  its  own  monitors. 

While  we  recognize  that  these  sponsors  had  other  prob¬ 
lems,  we  believe  effective  and  timely  monitoring  is  essential 
in  maintaining  quality  youth  programs. 


CHAPTER  3 


OPERATIONAL  PROBLEMS  HAMPERED 
PROGRAM  PERFORMANCE 

Sponsors  had  many  serious  operational  problems  in  imple¬ 
menting  Entitlement.  They  often  found  it  difficult  to  comply 
with  Labor  regulations  and  MDRC  directives  and  achieve  estab¬ 
lished  program  goals.  Some  problems  were  caused  by  the  rapid 
program  startup  and  were  resolved  early.  Other  problems  were 
more  persistent,  and  some  new  problems  surfaced  during  the 
second  year. 

The  operational  problems  were  so  severe  at  some  locations 
that  it  is  difficult  to  tell  how  the  program  would  have  been 
implemented  had  effective  management  been  employed.  Some  of 
the  more  difficult  problems  faced  by  sponsors  in  implementing 
Entitlement  were: 

— Entitlement  was  difficult  to  implement  because  of  its 
dual  role  as  a  demonstration  and  an  operating  program. 
As  the  program  progressed,  administrative  systems  often 
failed  to  meet  program  needs.  These  problems  and  high 
staff  turnover  led  to  major  reorganizations  and  program 
instability  at  most  Tier  I  locations. 

— Program  management  costs  were  higher  than  originally 
planned  and  averaged  about  37  percent  of  total  costs. 
The  major  cause  was  underenrollment  of  youths  coupled 
with  sponsor  reluctance  to  reduce  administrative 
staffs . 

— Sponsors  had  difficulty  in  enforcing  the  academic 
and  school  attendance  requirements  of  Entitlement. 

In  some  cases,  the  standards  were  so  liberal  as  to  be 
of  limited  value.  In  other  cases,  sponsors  did  not 
establish  the  necessary  processes  to  enforce  them 
consistently . 

— Sponsors  had  mixed  results  in  establishing  links 
and  improving  coordination  with  other  groups  which 
could  assist  them  in  better  serving  youths.  They 
often  found  it  difficult  to  establish  and/or  maintain 
cooperative  relationships  with  schools,  CBOs,  and 
other  agencies  serving  youths. 
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— Entitlement  is  the  first  youth  employment  program  which 
makes  extensive  use  of  subsidized  private  sector 
jobs.  However,  almost  half  of  the  jobs  are  the  type 
that  youths  are  normally  hired  for.  The  problem  of 
potential  substitution  is  compounded  by  the  fact  that 
most  sponsors  initially  provided  100-percent  subsidies 
to  all  employers,  including  those  in  the  private  sec¬ 
tor.  Sponsors  did  little  to  assure  that  the  Entitle¬ 
ment  participants  were  not  simply  a  substitute  for 
youths  who  would  have  been  hired  without  a  subsidy. 

ADMINISTRATIVE  PROBLEMS  AND  STAFF 
TURNOVER  LED  TO  MANY  REORGANIZATIONS 
AND  PROGRAM  INSTABILITY 

Entitlement,  because  of  its  dual  role  as  a  demonstra¬ 
tion  and  an  operational  program,  was  difficult  to  implement. 
Initially,  many  sponsors  could  not  cope  with  these  multiple 
responsibilities.  Delays  in  job  assignments  and  problems  in 
payroll  and  management  information  systems  were  caused  by 
the  large  influx  of  applicants. 

As  the  demonstration  progressed,  administrative  systems 
failed  to  meet  program  needs.  For  some  sponsors,  program 
suspensions  and  major  reorganizations  were  required.  Staff 
turnover  became  a  problem.  Although  some  problems  were 
resolved,  some  continued  into  1980. 

Baltimore  used  the  Summer  Youth  Employment  Program  as  a 
management  model  for  Entitlement.  Due  to  serious  management 
control  problems,  however,  this  model  proved  to  be  inadequate 
for  Entitlement.  As  a  result,  Baltimore  reorganized  twice 
during  the  first  year.  The  sponsor  could  not  cope  with  the 
paperwork  requirements  during  the  summer  of  1978.  CETA 
public  service  employees  used  for  administration  and  worksite 
monitoring  were  unqualified  for  the  tasks  and  further  compli¬ 
cated  the  problem. 

Paperwork  logjams  prevented  tracking  participant  assign¬ 
ments.  Transfers  of  job  assignments  were  not  documented, 
and  the  computer  rejected  timesheets  which  did  not  match 
recorded  worksites.  Thus,  hundreds  of  participants  were  not 
paid  on  time.  The  sponsor  suspended  program  operations  on 
two  occasions  and  used  the  time  to  correct  these  problems. 
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In  mid-1979,  a  third  reorganization  established  two  new 
units.  The  first  was  designed  to  monitor  worksites  and 
troubleshoot  operational  problems;  the  second,  to  monitor 
participants’  academic  and  attendance  performance.  At  the 
time  of  our  fieldwork,  Baltimore  still  did  not  have  an  effec¬ 
tive  system  for  monitoring  worksites  and  school  attendance 
and  academic  performance. 

Boston  had  numerous  problems  implementing  Entitlement. 

Its  large  centralized  administrative  structure  proved  inade¬ 
quate  in  coping  with  program  requirements.  The  job  assign¬ 
ment  system  faltered  from  the  program's  inception.  Hundreds 
of  youths  failed  to  receive  assignments  for  months.  In  April 
1979,  the  placement  unit  was  reorganized  and  the  backlog  of 
job  assignments  was  decreased.  However,  in  early  1980,  it 
still  took  about  6  weeks  for  a  youth  to  receive  a  job  assign¬ 
ment  after  enrollment.  Staff  turnover  disrupted  the  perform¬ 
ance  of  some  units.  The  caseworkers,  who  are  the  main  link 
between  the  worksite,  the  participant,  and  the  program's 
administration,  were  the  most  transient. 

Detroit  experienced  enrollment,  payroll,  and  information 
system  problems  so  severe  that  the  program  was  suspended  for 
2  months.  The  program  was  completely  reorganized  in  January 
1979.  MDRC  officials  told  us  this  program  has  turned  around. 
However,  as  of  December  1979,  some  of  the  original  problems 
still  existed.  Our  review  showed  that  participants'  files 
contained  inaccurate  information,  paychecks  were  late  or  in¬ 
accurate,  and  intake  centers  were  not  submitting  timely  status 
changes  on  participants. 

Cincinnati's  high  turnover  of  program  managers  contrib¬ 
uted  to  not  implementing  a  uniform  program  among  contractors. 
At  the  time  of  our  fieldwork,  three  individuals  had  served 
as  program  manager.  The  first  did  not  exert  a  strong  influ¬ 
ence  and  spent  minimal  time  with  the  contractors.  The  second 
failed  to  gain  needed  cooperation,  complained  of  a  lack  of 
authority,  and  resigned  after  5  months.  The  third  appeared 
to  be  taking  a  more  active  leadership  role. 

Denver  has  been  unable  to  implement  Entitlement  despite 
repeated  intervention  by  Labor  and  MDRC.  (See  pp.  49  and  50.) 
The  program  has  been  essentially  put  on  a  "life  support  sys¬ 
tem"  for  the  duration  of  the  demonstration.  In  its  first 
16  months,  the  program  had  four  directors  and  one  acting 
director.  According  to  MDRC,  with  each  change  Denver's 
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Entitlement  structure  also  changed,  so  the  program  never 
stabilized  long  enough  to  create  confidence  or  allow  any 
participating  agency  or  individual,  including  MDRC,  to 
clearly  understand  program  operations  in  Denver. 

Although  MDRC 1 s  March  1980  report  concludes  that  Entitle 
ment  is  on  the  whole  feasible,  it  cautions  against  rapid  im¬ 
plementation  of  future  programs  as  demanding  as  Entitlement. 
The  report  further  states  that: 

"*  *  *  [although]  substantial  improvements  in 
administration  are  noted,  *  *  *  the  program 
model  continues  to  pose  a  challenge  and  the 
effects  of  the  rapid  start-up  are  still 
apparent  at  some  sites." 

We  agree  that  improvements  have  been  made.  We  question, 
however,  whether  recurring  problems  2  years  into  the  program 
can  be  attributed  to  startup  difficulties. 

PROGRAM  MANAGEMENT  COSTS 
ARE  HIGHER  THAN  EXPECTED 

Most  other  CETA  programs  limit  the  percentage  of  funds 
which  sponsors  can  use  for  program  administration.  For  En¬ 
titlement,  on  the  other  hand,  MDRC  combined  program  adminis¬ 
tration  with  other  categories,  such  as  client  services  and 
worksite  supervision,  called  them  program  management  costs,  1 
and  placed  no  limit  on  them.  The  original  grant  agreements 
did,  however,  contain  approved  participant/management  cost 
ratios.  As  shown  below,  program  management  costs  as  a  per¬ 
centage  of  total  costs  were  originally  projected  to  average 
28  percent  at  Tier  I  locations.  From  the  program's  inception 
through  September  30,  1979,  management  costs  had  averaged 
37  percent,  or  about  one-third  higher  than  originally 
projected . 


1/Program  management  costs  include  all  costs  except  partici¬ 
pant  wages  and  fringe  benefits. 


Estimated  and  Actual  Program 
Management  Costs  as  a  Percentage 


of  Total 

Costs 

Percent 

Estimated  in 

estimated  in 

Actual  through 

original 

grant 

September  30, 

Sponsor 

grant 

extension 

1979 

Baltimore 

28 . 5 

39.5 

35.4 

Boston 

38.5 

41.2 

45.0 

Cincinnati 

29.8 

37.5 

39.0 

Denver 

28.3 

(a) 

42.1 

Detroit 

22.3 

39.6 

37.9 

Seattle 

30.9 

36.2 

42.5 

Mississippi 

23.1 

32.6 

25.6 

Average 

28.1 

38.1 

37.2 

a/Denver  did  not  have  an  approved  grant  extension  during  our 
fieldwork . 

As  shown  in  the  above  table,  higher  program  management 
costs  were  allowed  for  grant  extensions.  In  an  April  1979 
operations  memorandum  to  Entitlement  directors,  MDRC  was 
obviously  concerned  that  disproportionate  ratios  might  en¬ 
danger  the  acceptability  of  Entitlement  once  the  demonstra¬ 
tion  was  over.  In  this  regard  MDRC  stated: 

"We  should  all  understand  that  we  are  not  trying 
to  penalize  anyone,  but  only  trying  to  maintain  a 
demonstration  in  which  the  costs  and  management 
accountability  are  such  that  it  has  a  reasonable 
chance  of  being  accepted  as  an  on-going  policy 
vehicle. " 

MDRC  reported  that  through  August  1979,  Tier  II  sites 
devoted  an  even  higher  percentage  of  funds  to  management 
costs  (41  percent)  than  Tier  I  sites  (37  percent). 

The  variance  in  program  management  costs  among  the  spon¬ 
sors  together  with  a  variance  in  the  average  number  of  hours 
youths  actually  worked  is  reflected  in  the  following  full 
year  cost  estimates  for  participants  through  September  1979. 
Our  estimates  are  based  on  the  assumption  that  a  participant 
remains  in  Entitlement  for  a  full  year. 
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Total  annualized 
cost  per 


Sponsor 

participant 

Baltimore 

a/$3, 706 

Boston 

5,797 

Cincinnati 

4,621 

Denver 

5,294 

Detroit 

5,129 

Mississippi 

4,395 

Seattle 

5 , 164 

Average 

4,619 

a/Baltimore's  costs  are  lower  principally  because  it  limited 
participants'  work  to  15  hours  during  the  school  year  and 
30  hours  during  the  summer,  compared  to  other  sponsors, 
which  allowed  20  and  40  hours,  respectively. 

A  major  cause  of  high  program  management  cost  rates  was 
underenrollment  of  participants.  Sponsors  hired  management 
and  administrative  staffs  in  anticipation  of  enrollment  levels 
that  never  materialized.  MDRC  estimates  that  sponsors  were 
successful  in  enrolling  40  percent  of  the  eligible  youths  ir 
the  first  year  of  the  program.  According  to  MDRC,  most  spon¬ 
sors  in  their  initial  applications  overestimated  the  eligible 
population  in  order  to  receive  enough  funds  if  all  eligible 
youths  claimed  a  job. 

In  addition  to  the  overestimates  of  eligible  population, 
sponsors  may  have  overestimated  the  attractiveness  of  the 
program  and  underestimated  the  impact  of  competition  from 
other  youth  programs  which  do  not  require  a  commitment  to 
remain  in  or  return  to  school.  Strong  local  economies  in 
Seattle,  Denver,  and  Nashua  which  allowed  youths  to  obtain 
unsubsidized  employment  also  contributed  to  underenrollment. 

All  sponsors  faced  problems  in  recruiting  and  retaining 
out-of-school  participants.  About  10  percent  of  Entitlement 
participants  were  not  in  school  when  they  enrolled  in  the 
program.  Out-of-school  youths  may  not  be  attracted  to  a 
program  paying  the  minimum  wage,  with  work-hour  limitations 
of  20  hours  per  week  or  less  and  requiring  a  school  commit¬ 
ment.  Initially,  Boston  and  Mississippi  had  insufficient 
alternative  education  facilities  to  accommodate  those  drop¬ 
outs  who  enrolled  in  the  program.  This  deficiency  was 


overcome  through  the  establishment  of  additional  alternative 
education  programs. 

Retaining  former  dropouts,  even  in  an  alternative  school 
setting,  has  proven  difficult.  For  example,  Boston  enrolled 
1,200  dropouts.  As  of  December  1979,  500  were  attending  an 
alternative  education  program  or  a  regular  school.  Of  the 
700  youths  terminated,  only  44  (6  percent)  received  a  high 
school  diploma  or  its  eguivalent.  About  90  percent  of  all 
terminations  were  due  to  attendance  problems  or  lack  of 
interest.  Attendance  problems  and  lack  of  interest  in  alter¬ 
native  education  programs  are  not  unigue  to  Boston — several 
other  sponsors  experienced  the  same  problems. 

In  addition  to  enrollment  levels,  the  comprehensiveness 
of  services  also  affects  the  management  cost  rates.  As  dis¬ 
cussed  in  other  sections  of  this  report,  plans  to  counsel 
youths,  monitor  worksites,  and  enforce  academic  and  attendr 
ance  standards  were  not  fully  implemented.  To  the  extent 
that  these  services  are  upgraded  or  expanded,  management 
cost  rates  may  increase. 

DIFFICULTIES  IN  ASSURING  THAT  PARTICIPANTS 
MEET  ACADEMIC  AND  ATTENDANCE  STANDARDS 


A  unique  feature  of  Entitlement  is  that  the  job  guaran¬ 
tee  is  designed  to  be  an  inducement  for  economically  dis¬ 
advantaged  youths  to  stay  in  or  return  to  school  to  obtain 
a  high  school  diploma  or  its  equivalent.  Therefore,  as  a 
condition  of  an  Entitlement  job,  participants  must  meet  the 
minimum  academic  and  attendance  requirements  of  the  secondary 
school  or  high  school  eguivalency  program  in  which  they  are 
enrolled  or  be  terminated  from  the  program.  To  assure  that 
participants  attend  school,  academic  and  attendance  perform¬ 
ance  is  to  be  monitored  monthly. 

Initially,  some  school  systems  had  no  school  standards 
for  sponsors  to  monitor  and  enforce.  Therefore,  Labor  re-  ^ 
quired  that  sponsors  establish  standards  for  Entitlement 
participants.  Some  sponsors  established  standards  which 
were  more  stringent  than  existing  school  standards.  In  some 
cases,  students  had  to  pass  only  one  course.  In  other  cases, 
they  had  to  maintain  a  "C"  or  higher  average.  Attendance 
standards  also  varied  considerably.  See  appendix  III  for  a 
complete  list  of  academic  and  attendance  standards. 
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Enforcement  of  standards  was  inconsistent  and  often 
negligible.  In  March  1980  MDRC  reported  that  sponsors 
were  reluctant  to  terminate  youths  and  were  slow  to  enforce 
standards.  It  cited  several  factors  affecting  Entitlement 
sponsors'  termination  policies. 

— In  addition  to  operational  problems  in  establishing 
and \enforcing  standards,  sponsors  gave  low  priority 
to  this  area  because  of  other  systems'  problems. 

— Sponsors  were  inclined  to  maintain  enrollment  levels 
to  enhance  their  reputations. 

— Because  of  their  desire  to  help  poorly  performing 
youths,  sponsor  personnel  were  reluctant  to  terminate 
them. 

Further,  MDRC  reported  that  sponsors  were  struggling  to  main¬ 
tain  participant/management  cost  ratios.  Participant  termi¬ 
nations  could  adversely  affect  their  ratios  and  result  in 
cuts  in  management  funds. 

We  reviewed  available  school  records  for  participants 
interviewed,  but  included  only  those  who  had  been  in  the 
program  prior  to  the  summer  of  1979.  Data  on  22  percent 
were  either  unavailable  or  unusable.  For  the  remainder,  our 
review  showed  that  about  30  percent  of  active  participants 
were  not  meeting  school  performance  standards.  MDRC ' s 
Entitlement  information  system  shows  that  only  4  percent  of 
34,000  program  terminees  left  the  program  because  of  un¬ 
satisfactory  school  performance  through  December  1979. 

Sponsors  had  fairly  elaborate  systems  involving  warnings, 
appeals,  and  hearings  for  participants  not  meeting  school 
standards.  Baltimore  and  Boston  also  placed  their  own  staff 
as  liaisons  in  the  schools  to  facilitate  the  collection  of 
school  performance  data.  Although  these  sponsors  terminated 
youths  more  frequently  than  other  sponsors,  their  systems 
were  often  ineffective. 

Boston  liaisons  recommended  terminating  466  youths  for 
inadequate  school  performance;  however,  only  64  were  termi¬ 
nated.  These  466  youths  had  received  two  warning  notices 
and  a  notification  of  pending  termination.  Our  sample  of 
Boston  participants  included  one  participant  who,  although 
enrolled,  did  not  attend  school  the  entire  1978-79  school 
year  but  worked  in  an  Entitlement-sponsored  job.  Four  other 
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participants  missed  from  111  to  167  days  (62  to  93  percent) 
of  the  180-day  school  year,  but  none  were  terminated.  These 
absence  rates  far  exceeded  the  45  days  per  academic  year 
Boston  allowed  for  unexcused  absences.  In  addition,  these 
participants  did  not  meet  the  minimum  academic  requirements. 

Although  Baltimore  had  terminated  a  higher  percentage  of 
youths  for  not  meeting  school  standards  than  other  sponsors, 
its  system  needed  improvement.  The  sponsor's  academic  com¬ 
pliance  unit  was  extremely  disorganized.  Personnel  did  not 
understand  their  function  or  the  system.  We  reviewed  the 
files  for  164  Baltimore  participants  and  found  that  many 
lacked  the  data  necessary  for  evaluating  a  participant's 
school  status.  Other  sponsors  did  even  less  in  enforcing 
school  performance  standards.  Detroit  and  Seattle  were  the 
lowest  of  Tier  I  sponsors  with  less  than  1  percent  of  their 
terminations  for  this  reason. 

COORDINATION  BETWEEN  PRIME  SPONSORS  AND  OTHER 
ORGANIZATIONS  SERVING  YOUTHS  NEEDS  IMPROVEMENT 

YEDPA  required  that  sponsors  establish  links  with 
schools,  CBOs ,  and  other  agencies  serving  youths.  All  spon¬ 
sors  we  visited  made  some  progress  in  obtaining  commitments 
from  local  institutions  and  organizations.  However,  a  number 
of  important  relationships  were  not  always  lasting  or  without 
conflict.  At  some  locations,  this  inability  to  work  coopera¬ 
tively  caused  significant  problems. 

Sponsors'  relationships  with  schools 

By  design.  Entitlement  required  a  special  relationship 
between  sponsors  and  schools.  Schools  provided  easy  access 
to  potential  participants,  were  essential  to  providing  spon¬ 
sors  with  academic  and  attendance  data  on  participants,  and 
could  assist  sponsors'  job  development  efforts  by  arranging 
flexible  school  schedules  for  participants.  At  some  loca¬ 
tions,  schools  played  an  important  role  in  managing  the 
program . 

Detroit  initially  contracted  with  the  school  system  to 
implement  the  program.  As  discussed  further  on  page  50,  this 
relationship  deteriorated  to  the  point  where  Labor  and  MDRC 
required  a  suspension  of  program  operations  for  2  months. 
Later,  the  sponsor  assumed  responsibility  for  managing  the 
program. 
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In  Denver,  the  sponsor  and  the  school  system  discontinued 
their  relationship  during  contract  renegotiations.  The  school 
department  terminated  the  relationship  because  school  offi¬ 
cials  did  not  believe  their  funding  level  was  commensurate 
with  their  responsibilities.  This  was  one  issue  contributing 
to  the  Labor  and  MDRC  decision  to  suspend  new  enrollments 
in  Denver  from  June  1979  to  at  least  June  1980.  (See  p.  49.) 


As  previously  stated,  most  Tier  I  sponsors  had  major 
problems  in  obtaining  complete  and  timely  data  on  the  academic 
performance  and  attendance  of  participants.  Also,  the  spon¬ 
sors  were  not  very  successful  in  securing  early  releases  from 
school  for  Entitlement  participants.  This  affected  job  de¬ 
velopment  efforts  in  Cincinnati,  Baltimore,  Detroit,  Seattle, 
and  Mississippi.  Some  of  these  sponsors  were  able  to  secure 
early  releases  for  participants  on  a  case-by-case  basis,  but 
generally  not  on  a  large  scale. 

Sponsors'  relationships  with 
CBOs  and  other  agencies 

All  Tier  I  sponsors  established  relationships  with  CBOs, 
but  some  important  ones  were  strained  to  the  point  of  ter¬ 
mination.  In  some  cases,  CBOs  were  dissatisfied  with  their 
roles,  while  in  others,  sponsors  were  unhappy  with  CBOs' 
performance.  These  problems  had  significant  impacts  on  im¬ 
plementing  Entitlement  at  some  locations. 

Boston  and  a  major  CBO  never  resolved  serious  differ¬ 
ences  between  them.  According  to  CBO  officials,  the  sponsor 
had  led  them  to  believe  the  CBO  would  play  a  major  role  in 
operating  the  program.  In  its  grant  proposal,  the  sponsor 
stated  that  the  CBO  would  participate  in  intake,  job  develop¬ 
ment,  and  counselor  training  for  the  duration  of  the  program. 
In  January  1979,  the  sponsor  reduced  the  CBO's  responsibili¬ 
ties  to  the  outreach  and  intake  of  out-of-school  youths.  The 
sponsor  and  the  CBO  constantly  disagreed  and  exchanged  accu¬ 
sations  during  the  19-month  contract  period.  The  sponsor 
claimed  the  CBO  was  not  meeting  its  recruiting  goals,  while 
the  CBO  claimed  that  the  sponsor  took  too  long  getting  jobs 
for  youths  it  referred.  The  situation  also  deteriorated 
because  the  CBO  failed  to  fulfill  its  commitment  of  providing 
jobs  under  another  program  to  Entitlement  eligible  youths 
and  graduates  for  the  summers  of  1978  and  1979,  respectively. 

Mississippi  and  two  of  its  major  contractors,  the  Uni¬ 
versity  of  Southern  Mississippi  and  the  State  Employment 


Security  Commission,  had  serious  problems  of  cooperation  and 
coordination.  Under  its  contract  with  the  sponsor,  the  uni¬ 
versity  was  to  provide  services,  such  as  transportation, 
alternative  education,  and  the  supervision  of  counselors. 

It  was  also  to  coordinate  these  activities  with  the  school 
districts.  In  addition,  the  university  was  responsible  for 
establishing  a  management  information  system.  The  university 
did  not  perform  satisfactorily,  and  the  contract  was  not  re¬ 
newed  when  it  expired  in  June  1979.  Mississippi  then  nego¬ 
tiated  contracts  with  four  CEOs  and  individual  school  dis¬ 
tricts  to  perform  these  functions. 

Mississippi  and  the  State  Employment  Security  Commission, 
which  was  responsible  for  jot  development,  were  often  at  odds. 
The  sponsor  stated  that  the  egency  was  not  diligent  in  devel¬ 
oping  jobs.  The  agency  stated  that  inadequate  transportation 
and  inflexible  school  schedules  were  the  real  causes  for  the 
lack  of  jobs. 

Denver's  relationship  with  two  nationally  recognized 
CBOs  ended  in  failure.  Under  their  contracts  with  the  spon¬ 
sor,  the  CBOs  were  to  recruit  1,100  out-of-school  youths  and 
enforce  alternative  education  attendance  standards.  The  CBOs 
fell  far  short  of  their  recruitment  goals,  and  class  attendance 
standards  were  not  enforced.  Dissatisfied  with  this  perform¬ 
ance,  Denver  did  not  renew  their  contracts  when  they  expired 
in  June  1979.  Denver  did  not  have  a  contract  with  anyone 
to  handle  out-of-school  youths  until  February  1980,  when  it 
negotiated  a  contract  with  a  State  university. 


ACHIEVING  A  NET  INCREASE  IN  PRIVATE 
SECTOR  JOBS  FOR  YOUTHS  IS  DIFFICULT 


YEDPA  requires  that  Entitlement  jobs  not  result  in  sub¬ 
stituting  currently  employed  workers  by  program  youths.  As 
the  program  progressed,  total  private  sector  hours  worked 
increased  from  14  percent  in  September  1978  to  23  percent  in 
January  1980.  While  private  sector  jobs  are  generally  per¬ 
ceived  as  better  work  experiences  for  youths,  there  is  po¬ 
tential  for  substitution  because  of  the  subsidy  used  to 
induce  employers  to  participate. 

The  act  allowed  sponsor^  to  pay  various  subsidies  up 
to  100  percent  of  the  wages  for  youths  employed  in  the 
private  sector,  without  limiting  how  long  the  subsidy  was 
to  be  paid.  This  provision  permitted  Labor  to  test  what 
subsidy  levels  are  necessary  to  encourage  private  employer 


participation  in  the  program.  With  the  exception  of 
Mississippi  at  75  percent,  all  Tier  I  sponsors  initially 
developed  private  sector  jobs  providing  for  100-percent 
subsidy. 

Although  we  did  not  conduct  an  indepth  review  of  the 
extent  that  Entitlement  participants  displaced  currently 
employed  workers,  we  found  evidence  indicating  that  Labor 
and  the  sponsors  need  to  more  closely  monitor  the  payment  of 
wage  subsidies.  Of  approximately  4,000  private  sector  jobs 
held  by  youths  during  our  fieldwork,  we  estimate  that  about 
45  percent  were  with  employers  who  normally  employ  youths 
for  these  jobs.  This  estimate  is  based  on  our  review  of 
sponsor  job  lists,  interviews  with  supervisors  and  partici¬ 
pants,  and  observations  at  worksites.  The  following  examples 
indicate  the  problems  with  wage  subsidies. 

Detroit  used  many  fast  food  restaurants  as  worksites. 
Several  fast  food  managers  told  us  that  they  treat  Entitle¬ 
ment  participants  as  regular  employees  and  use  them  inter¬ 
changeably  with  employees  they  pay.  Although  the  sponsor 
monitored  for  substitution,  we  found  evidence  of  substitution 
at  10  of  26  private  sector  worksites.  During  our  fieldwork, 
Detroit  was  expanding  the  use  of  fast  food  restaurants,  thus 
increasing  the  possibility  of  additional  substitution. 

In  Boston,  we  found  substitution  at  two  of  five  private 
employers  who  normally  hire  youths.  Both  employers  told  us 
that  their  previous  employees  were  about  the  same  age  as  the 
Entitlement  participants.  Although  one  employer  had  hired 
an  Entitlement  participant  before  the  departure  of  his  regular 
youth  employee,  he  replaced  the  regular  employee  with  another 
Entitlement  participant.  The  other  employer  not  only  replaced 
his  regular  youth  employee  with  an  Entitlement  participant, 
but  hired  three  other  Entitlement  participants  to  do  the  same 
work  that  one  regular  employee  did.  Both  employers  stated 
that  they  would  hire  unsubsidized  employees  if  the  Entitle¬ 
ment  participants  were  not  available. 

In  Mississippi,  14  of  20  employers  who  normally  hire 
youths  stated  they  would  hire  an  unsubsidized  employee  if 
the  Entitlement  program  was  not  available.  Three  employers 
had  Entitlement  eligible  youths  working  for  them  when  they 
became  Entitlement  job  sponsors.  The  youths  enrolled  in  the 
Entitlement  program  and  continued  working  at  the  same  jobs. 

One  youth  stated  that  his  employer  required  him  to  enroll  in 
Entitlement  or  be  fired.  Thus,  Entitlement  is  paying  the 
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salaries  of  youths  which  were  previously  paid  by  their 
employers . 

In  addition  to  private  sector  substitution,  the  issue 
of  achieving  variation  in  the  subsidy  level  was  not  easily 
resolved.  As  early  as  the  first  grant  awards  in  1978,  Labor 
and  MDRC  required  sponsors  to  submit  subsidy  reduction  plans. 
This  requirement  was  embodied  in  Entitlement  regulations 
issued  in  October  1979.  Subsidy  reduction  plans  were  time- 
phased  reductions  from  the  100-percent  subsidy  levels  most 
sponsors  were  using.  Thus,  after  a  youth  was  assigned  to 
a  worksite  for  a  particular  length  of  time,  6  or  12  months, 
the  private  employer  was  to  reimburse  the  sponsor  for  part 
of  the  wages  or  lose  the  participants. 

Generally  sponsors  resisted  these  attempts,  particularly 
Baltimore  and  Boston.  They  argued  that  (1)  the  100-percent 
subsidy  was  necessary  to  obtain  private  sector  participation 
and  (2)  they  had  negotiated  private  sector  jobs  in  good  faith 
at  the  100-percent  level  and  believed  it  unfair  to  renegotiate 
these  agreements  after  the  fact.  Baltimore  finally  imple¬ 
mented  a  subsidy  reduction  plan  in  the  fall  of  1979.  In 
February  1980,  MDRC  reported  that  “the  plan  continues  to  be 
implemented  but  without  much  success."  Boston  did  not  imple¬ 
ment  its  wage  subsidy  reduction  plan  until  March  1980. 

Detroit  was  more  successful  in  implementing  subsidy 
reduction.  It  began  implementing  its  plan  in  August  1979 
and  required  employers  who  had  employed  Entitlement  partici¬ 
pants  for  a  6-month  period  since  January  1979  to  agree  to  a 
25-percent  subsidy  reduction  or  lose  the  participant  and 
the  slot.  As  of  January  1980,  this  subsidy  reduction  plan 
affected  118  employers  with  249  participants.  Only  23  em¬ 
ployers  with  29  participants  rejected  this  plan. 

After  much  discussion  and  negotiation,  most  sponsors 
have  not  required  very  many  private  employers  to  absorb  any 
costs  of  their  Entitlement  participants.  In  this  regard, 

MDRC  stated  in  its  May  1980  monthly  report  that  subsidy 
reduction  plans  are  having  a  minimal  impact  in  terms  of 
actual  dollars  collected  from  private  employers.  Further, 
it  reported  that  many  sponsors  are  transferring  youths  to 
other  worksites  rather  than  keeping  them  at  their  original 
ones  at  reduced  subsidy  levels,  thus  allowing  for  the  con¬ 
tinuation  of  100-percent  subsidies. 
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Entitlement  was  the  first  large-scale  test  of  the  use 
of  wage  subsidies  in  the  private  sector.  It  is  probably 
inevitable  that  some  substitution  or  displacement  would 
occur  in  a  job  creation  program  of  this  size,  particularly 
one  providing  a  100-percent  subsidy.  If,  however,  subsidies 
to  the  private  sector  are  to  be  incorporated  in  future  youth 
initiatives,  much  more  thought  must  go  into  the  methodology 
for  implementing  and  monitoring  their  use. 
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CHAPTER  4 


INTENSIVE  OVERSIGHT  REQUIRED 
TO  IMPLEMENT  ENTITLEMENT  AS  A  DEMONSTRATION 


In  chapters  2  and  3,  we  discussed  Entitlement's  opera¬ 
tional  aspects  and  its  program  impact.  This  chapter  discusses 
Entitlement  as  a  demonstration.  Since  Entitlement's  results 
might  have  a  major  influence  on  the  approach  this  Nation  will 
take  in  dealing  with  a  problem  that  has  resisted  solution, 
it  is  important  that  any  conclusions  regarding  the  program 
be  based  on  objective  information  derived  from  a  valid 
demonstration . 

Labor  has  undertaken  a  major  research  effort  to  answer 
questions  raised  by  the  Congress  about  Entitlement’s  impact, 
implementation,  and  cost.  From  the  beginning  of  Entitlement, 
poor  recordkeeping  at  the  sponsor  level  caused  problems  in 
providing  complete,  accurate,  and  timely  data  needed  for  re¬ 
search.  As  a  result,  MDRC  made  major  remedial  efforts  so 
that  the  data  needed  for  research  can  be  used.  MDRC  is  con¬ 
fident  that  these  efforts  have  resulted  in  reliable  research 
data . 


Labor  and  MDRC  did  not  allow  the  demonstration  to 
succeed  or  fail  in  a  normal  environment.  MDRC  had  to  inter¬ 
cede  in  program  operations  at  most  Tier  I  locations  and  at 
one  of  the  Tier  II  sites  we  reviewed.  By  providing  inten¬ 
sive  oversight,  monitoring,  technical  assistance,  and  in  some 
cases  direct  intervention  at  a  level  not  normally  accorded 
CETA  programs.  Labor  and  MDRC  were  in  a  unique  position  to 
have  a  positive  influence  on  program  results.  At  some  spon¬ 
sors,  this  special  attention  greatly  improved  program  opera¬ 
tions;  at  others,  it  salvaged  marginal  operations. 

THE  CONGRESS  REQUIRES  DETAILED 
INFORMATION  ON  ENTITLEMENT 


The  Congress  recognized  the  need  for  detailed  information 
on  Entitlement  to  aid  it  in  making  policy  decisions  regarding 
future  youth  employment  programs.  In  this  regard,  YEDPA 
specifically  required  that  Labor  report  on  the  following: 


1.  The  number  of  enrolled  youths. 

2.  The  cost  of  providing  jobs. 


3.  The  degree  to  which  jobs  caused  out-of-school  youths 
to  return  to  school  or  others  to  remain  in  school. 

4.  The  number  of  youths  provided  jobs  in  relation  to 
the  total  which  might  have  been  eligible. 

5.  The  kinds  of  jobs  provided  and  a  description  of  em¬ 
ployers  . 

6.  The  degree  to  which  on-the-job  or  apprenticeship 
training  was  provided  as  part  of  employment. 

7.  The  estimated  cost  of  such  a  program  if  it  were  to 
be  extended  to  all  areas. 

8.  The  effect  jobs  had  on  reducing  youth  unemployment 
in  the  areas  served  by  the  Entitlement  program. 

9.  The  impact  of  Entitlement  jobs  on  other  job  oppor¬ 
tunities  in  the  area. 

Labor  inferred  two  additional  research  issues  from  the  lan¬ 
guage  of  YEDPA :  first,  the  degree  to  which  youths  receive  a 
quality  work  experience;  and  second,  the  efficacy  of  differ¬ 
ent  administrative  mechanisms  in  implementing  Entitlement. 

To  answer  these  issues,  Labor  engaged  MDRC  to  manage  a 
large-scale  research  effort.  This  effort  seeks  to  determine 
how  well  Entitlement  accomplishes  what  it  is  designed  to 
achieve  and  whether  the  program  should  be  extended  and  ex¬ 
panded  nationally.  MDRC  designed  a  three-part  research 
program  to  analyze  Entitlement's  impact,  implementation, 
and  costs.  MDRC  subcontracted  with  Abt  Associates,  Inc., 
to  perform  much  of  the  impact  analysis. 

MDRC  designed  the  impact  analysis  to  measure  partici¬ 
pation  rates  of  eligible  youths  and  to  assess  Entitlement's 
impact  on  labor  force  participation,  employment  and  earnings, 
and  school  attendance  and  completion.  Abt  is  obtaining  data 
for  this  analysis  from  four  waves  of  interviews  with  a  random 
household  sample  of  eligible  youths  and  their  parents  in  the 
Baltimore,  Cincinnati,  Denver,  and  part  of  Mississippi's 
rural  Entitlement  areas.  Abt  is  also  obtaining  data  from 


i 


four  matched  non-Entitlement  locations  that  will  serve  as 
control  locations.  Respectively,  the  locations  are  Cleve¬ 
land,  Louisville,  Phoenix,  and  part  of  Mississippi's  non- 
Entitlement  area.  These  control  locations  match  the  four 
Entitlement  locations  in  geographic  and  economic  character¬ 
istics  and  are  being  statistically  sampled  in  the  same  way 
as  the  Entitlement  locations.  Abt  plans  to  use  the  data 
from  the  control  locations  to  assess  the  extent  to  which  any 
changes  in  youths'  educational  and  employment  behavior  can 
be  attributed  to  Entitlement. 

Using  data  from  the  first  wave  of  interviews  conducted 
in  the  spring  and  early  summer  of  1978,  Abt  completed  an 
analysis  of  the  characteristics  of  youths  eligible  for 
Entitlement.  To  respond  to  Labor's  request  for  early  results 
regarding  program  impact  on  school  enrollment,  Abt  and  MDRC 
prepared  an  interim  report  which  provides  a  preliminary  esti¬ 
mate  of  Entitlement's  early  impact  on  school  retention  and 
the  return  to  school  by  dropouts.  The  report  did  not  provide 
definitive  results  because  it  was  based  on  limited  data, 
which  Abt  considered  seriously  flawed.  MDRC  expects  later 
analyses  to  provide  more  definitive  results  and  plans  to 
issue  the  final  impact  analysis  report  in  1982. 

MDRC  designed  the  cost  analysis  to  measure  program  costs 
and  estimate  the  cost  of  a  nationwide  Entitlement  program. 
MDRC  has  pointed  out  to  Labor  that  getting  beyond  gross  cost 
measurements  may  be  impossible  due  to  the  nature  of  the  CETA 
funding  system  and  the  difficulty  in  getting  detailed  timely 
and  accurate  cost  breakdowns  from  Entitlement  locations.  For 
the  cost  analysis,  MDRC  is  relying  on  data  from  Entitlement's 
fiscal  reporting  and  information  systems.  MDRC  will  use  data 
from  these  systems  to  determine  total  program  costs,  costs 
per  participant,  and  costs  per  participant  year  for  different 
subgroups  of  youths. 

In  testifying  before  the  Subcommittee  on  Employment 
Opportunities  of  the  House  Education  and  Labor  Committee  in 
March  1980,  an  MDRC  official  provided  the  following  cost 
projections  for  Entitlement  if  it  is  continued  through 
fiscal  year  1981.  Using  existing  eligibility  criteria, 

MDRC  estimated 

— $85  million  to  continue  at  the  existing  sites, 

— $200  million  to  expand  to  20  additional  sites 
(1  Tier  I  and  1  Tier  II  in  each  Labor  region). 


— $485  million  to  expand  to  all  designated  poverty 
areas,  or 

— $1,173  billion  to  expand  to  all  eligible  youths. 

MDRC  designed  the  third  part  of  the  research  program, 
the  implementation  analysis,  to  describe  and  assess  the 
Entitlement  demonstration's  operational  development  at 
various  locations.  It  will  attempt  to  assess  the  feasibil¬ 
ity  of  providing  guaranteed  jobs  and  to  identify  factors 
that  affected  program  operations.  The  general  research  on 
Entitlement's  implementation  covers  certain  key  areas, 
including 

— a  description  of  program  content  and  operations, 

— implementation  factors  affecting  enrollment  and  con¬ 
tent  and  continued  participation, 

— relationships  between  prime  sponsors  and  schools, 

— the  private  sector's  role  in  the  demonstration, 

— implementing  the  program  in  rural  areas, 

— Tier  II  innovations,  and 

— the  quality  of  worksites. 

Data  for  the  implementation  research  come  from  three  primary 
sources:  (1)  the  Entitlement  information  system,  (2)  MDRC 

field  operations  staff,  including  onsite  monitors  at  Tier  I 
locations,  and  (3)  interviews  with  program  officials  and 
others  knowledgeable  of  program  operations.  As  of  March 
1980,  MDRC  had  issued  two  interim  reports  on  program  imple¬ 
mentation  and  costs. 

In  addition  to  the  three-part  research  program,  MDRC 
has  undertaken  special  studies  which  will  address  worksite 
quality,  program  implementation  in  rural  areas,  private 
sector  participation,  and  labor  market  effects.  In  addition 
to  a  series  of  reports  on  specific  issues  and  additional 
interim  reports,  MDRC  expects  to  issue  a  final  summary 
report  on  Entitlement  in  1982 . 


TO  CORRECT  DATA  PROBLEMS 


The  importance  of  valid  and  meaningful  data  regarding 
the  Entitlement  demonstration  cannot  be  overemphasized.  To 
fulfill  their  research  responsibilities,  Labor  and  MDRC  need 
complete,  accurate,  and  timely  data  on  each  participant  as 
well  as  information  on  their  Entitlement  job.  MDRC  relies 
on  sponsors  to  provide  these  data. 

We  have  not  made  a  detailed  review  of  the  data  problems 
encountered  on  the  Entitlement  demonstration  nor  are  we  in  a 
position  to  say  whether  or  not  these  problems  could  affect 
the  research  results.  However,  information  gathered  during 
our  review  shows  that  most  sponsors  experienced  problems  in 
providing  complete,  accurate,  and  timely  Entitlement  data 
needed  for  MDRC's  research,  especially  for  the  cost  and  im¬ 
plementation  analyses.  MDRC  is  confident  that  its  cleanup 
efforts  have  resulted  in  reliable  research  data. 

In  setting  up  its  research  design,  MDRC  concluded  that 
the  CETA  information  system  used  by  sponsors  was  not  suffi¬ 
cient  for  the  needs  of  Entitlement.  Although  the  CETA  system 
has  standardized  reporting  requirements,  it  does  not  have 
standardized  forms  to  collect  data.  CETA  allows  sponsors  to 
develop  their  own  forms  tailored  to  their  individual  needs. 
MDRC  needed  data  on  Entitlement  participants'  school  and  em¬ 
ployment  history,  and  the  CETA  system  was  not  designed  to 
obtain  this.  Therefore,  MDRC  designed  an  Entitlement  infor¬ 
mation  system  to  collect  standardized  data  among  sponsors, 
to  minimize  additional  data  collection  burdens,  and  to  pro¬ 
vide  compatibility  with  the  CETA  system. 

As  discussed  in  chapter  1  (see  p.  6),  the  capability  of 
a  sponsor  to  meet  Entitlement  information  needs  was  a  major 
criterion  in  the  selection  process.  Labor's  grant  agreement 
with  the  chosen  sponsors  included  a  copy  of  the  Entitlement 
information  system  specifications.  MDRC  prepared  a  detailed 
instruction  manual,  distributed  it  to  each  participating 
sponsor,  and  provided  a  training  course  on  the  information 
system.  MDRC  continued  onsite  training  activities  as  needed. 
Although  it  had  originally  envisioned  having  onsite  MDRC  in¬ 
formation  system  staff  at  each  Tier  I  location,  MDRC  abandoned 
this  plan  for  cost  and  efficiency  reasons. 
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MDRC  has  reported  that  the  effectiveness  of  data  collec¬ 
tion  efforts  is  generally  dependent  on  the  following  factors: 

"1.  \the  experience,  skill,  and  degree  of  turnover 
of  the  local  intake,  fiscal,  and  MIS  [manage¬ 
ment  information  system]  staff; 

2.  the  intensity  of  youth  enrollment  and  the 
level  of  staffing  provided  to  handle  the 
flow; 

3.  the  level  of  quality  control  established 
locally  to  examine  documents  before  their 
submission  to  MDRC; 

4.  the  general  preparedness  of  the  prime  spon¬ 
sor  to  handle  the  data  collection  function 
smoothly  and  completely  from  the  onset  of  the 
demonstration;  and 

5.  the  overall  commitment  of  participating  prime 
sponsors  to  provide  high-quality,  timely 
data . " 

Poor  recordkeeping 

MDRC  requires  participating  sponsors  to  submit  data  for 
each  participant  and  job.  Supporting  files  must  include 
copies  of  eligibility  records,  status  change  records,  and 
various  other  records  which  are  the  supporting  documentation 
for  the  Entitlement  information  system.  After  interviewing 
participants,  we  reviewed  their  files  to  determine  whether 
they  contained  the  required  documentation.  Generally  they 
did . 


For  example,  99  percent  of  the  files  contained  the 
required  enrollment  records  which  document  demographic  and 
background  information  about  each  participant.  Also,  99  per¬ 
cent  of  the  participants'  files  contained  documentation  showing 
that  the  participant  was  economically  disadvantaged,  98  per¬ 
cent  had  the  documentation  indicating  that  the  participant 
had  been  certified  as  eligible,  and  88  percent  contained  the 
required  documentation  showing  that  the  participant  was  in 
school . 
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We  also  checked  the  files  to  determine  whether  they 
accurately  showed  the  status  of  the  participants  we  inter¬ 
viewed.  MDRC  requires  sponsors  to  prepare  status  change 
forms  to  document  three  types  of  status:  job  assignment, 
hold,  and  terminations.  However,  only  68  percent  of  the 
files  accurately  recorded  a  participant's  program  status. 

In  its  April  1979  interim  implementation  report,  MDRC  re¬ 
ported  problems  with  participant  status  records  and  recog¬ 
nized  that  its  Entitlement  information  system  data  were  in¬ 
complete,  inaccurate,  and  untimely  in  this  regard. 

Certain  Tier  I  sponsors  experienced  major  management 
information  and  payroll  problems  requiring  the  temporary 
suspension  of  some  programs.  MDRC,  recognizing  the  lack  of 
valid  program  data,  intervened  to  help  resolve  the  problems. 
More  intensive  intervention,  however,  was  required  at  some 
sponsors.  (See  pp.  49  to  51.)  in  spite  of  these  efforts, 
the  problems  persisted  at  some  locations.  For  example: 

— In  August  1978,  MDRC  found  that  Baltimore  had  manage¬ 
ment  information  problems  which  included  missing 
forms  and  participant  files,  duplicate  enrollments, 
the  failure  to  forward  status  change  forms  to  MDRC, 
a  98-percent  error  rate  1/  for  forms  that  were  for¬ 
warded,  and  poor  worksite  records.  MDRC  also  found 
in  August  1978  that  Baltimore  had  conducted  little 
management  information  training  for  its  staff  and 
questioned  the  qualifications  of  some.  Baltimore 
had  an  information  system  which  MDRC  described  in 
November  1978  as  being  "quite  slow  and  error  ridden" 
and,  therefore,  it  could  not  be  used  for  its  report 
to  Labor  and  the  Congress.  In  October  1979,  MDRC 
criticized  the  sponsor  for  slow  error  correction  and 
late  submission  of  performance  and  fiscal  reports. 

In  addition,  we  found  that  the  sponsor  often  lacked 
participant  academic  and  attendance  information. 

— According  to  MDRC,  Boston  had  resolved  early  informa¬ 
tion  system  problems  to  the  extent  that  MDRC  found 
very  little,  if  anything,  to  criticize  and  concluded 
that  the  system  seemed  to  be  in  excellent  shape  in 
September  1979.  Later,  after  reviewing  payroll 

l/ln  commenting  on  our  draft  report,  Baltimore  asserted  that 
the  98-percent  error  rate  was  for  1  week  and  was  not  a 
normal  condition. 
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records  and  other  information  system  data,  we  were 
unable  to  trace  the  program  history  of  19  of  78  par¬ 
ticipants.  We  reported  to  the  sponsor  that  during 
our  worksite  visits  we  had  interviewed  two  partici¬ 
pants  whose  records  showed  that  they  had  been  ter¬ 
minated.  As  a  result,  Boston  initiated  a  review  of 
4,900  records  which  revealed  that  over  1,500  youths 
continued  to  work  after  they  had  been  terminated 
from  Entitlement.  For  the  period  these  participants 
worked  after  termination,  terminations  were  over¬ 
stated.  MDRC  requested  a  certified  public  accounting 
firm  to  review  the  sponsor's  internal  controls  and 
systems  flow.  At  MDRC ' s  insistence,  in  March  1980 
the  sponsor  froze  the  processing  of  applications  and 
transfers  to  allow  staff  to  concentrate  on  a  massive 
cleanup  of  the  information  system.  No  new  applica¬ 
tions  were  accepted  for  10  weeks  during  this  process. 

— Cincinnati  was  consistertly  delinquent  in  submitting 
required  data.  At  one  point,  it  was  10  months  behind 
schedule  in  submitting  monthly  performance  reports, 
and  MDRC  described  the  one  participant  roster  that 
had  been  submitted  as  so  incomplete  that  it  was  use¬ 
less.  The  sponsor  had  to  revise  its  monthly  operat¬ 
ing  reports  for  April  1978  to  March  1979  due  to  ex¬ 
penditure  discrepancies.  However,  even  with  the 
revisions,  we  had  difficulty  reconciling  the  data 
from  one  report  to  the  next. 

While  each  of  these  examples  related  to  a  particular 
Tier  I  sponsor,  management  information  problems  were  wide¬ 
spread.  For  example,  each  Tier  I  and  some  Tier  II  sponsors 
had  problems  in  obtaining  and  submitting  data  on  school 
attendance.  Also,  as  of  November  1979,  MDRC's  Entitlement 
information  data  showed  that  youths  assigned  to  a  job  and 
not  working  ranged  from  10  to  47  percent,  averaging  23  per¬ 
cent  for  the  seven  Tier  I  prime  sponsors.  We  recognize  that 
there  may  be  legitimate  reasons  why  assigned  youths  would 
not  be  working,  such  as  work  absences,  but  if  they  are  not 
working  for  extended  periods,  the  sponsor  should  place  them 
in  a  hold  or  terminated  status. 

After  the  completion  of  our  fieldwork,  MDRC  officials 
stated  that,  in  spite  of  the  recordkeeping  problems  at  the 
sponsor  level,  they  were  confident  the  data  were  sufficiently 
reliable  for  research  purposes.  MDRC  told  us  that,  in  sum¬ 
mary,  it  has  three  distinct  levels  of  quality  control:  first, 
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it  checks  each  document  and  report  before  entering  it  into 
the  computer;  second,  it  identifies  errors  of  logic;  and 
third,  it  reconciles  Entitlement  information  system  data  with 
fiscal  reports  and  telephone  call-in  data.  In  addition,  MDRC 
also  conducts  periodic  reviews  and  requires  corrective  action 

ENTITLEMENT  WAS  NOT  A 
REAL  WORLD  DEMONSTRATION 


Labor  regulations  state  that  the  basic  purpose  of  En¬ 
titlement  Tier  I  projects  is  to  test  the  experimental  idea 
of  a  guaranteed  job  by  determining  whether  sponsors  can  fea¬ 
sibly  implement  substantial  programs.  Tier  II  projects  are 
to  test  specific  Entitlement  innovations.  To  be  a  valid 
demonstration,  we  believe  sponsors  should  conduct  the  pro¬ 
gram  in  a  normal  environment,  that  is,  one  similar  to  that 
in  which  they  operate  other  CETA  programs.  Labor  and  MDRC 
did  not  permit  a  real  world  test  of  Entitlement's  success 
or  failure.  Due  to  widespread  problems,  they  interceded  to 
salvage  some  marginal  operations  by  providing  intensive  over¬ 
sight,  monitoring,  and  technical  assistance.  Only  rarely, 
however,  are  sponsors  accorded  the  degree  of  oversight,  mon¬ 
itoring,  and  technical  assistance  received  by  the  Entitle¬ 
ment  sponsors. 

Normally,  Labor  regional  representatives  assigned  to 
each  sponsor  are  the  focal  point  through  which  the  spon¬ 
sors'  effectiveness  is  evaluated.  Their  monitoring  activ¬ 
ities  for  all  CETA  programs  operated  by  that  sponsor  con¬ 
sist  of  (1)  routine  contacts  with  sponsor  officials  either 
by  telephone  or  occasional  site  visits,  (2)  desk  reviews  and 
evaluations  of  quarterly  reports,  (3)  an  annual  assessment 
of  sponsor  performance,  and  (4)  reviews  to  determine  whether 
sponsors  are  complying  with  CETA's  requirements.  Time  and 
workload  constraints  do  not  permit  Labor  regional  representa¬ 
tives  to  continuously  monitor  programs  in  the  manner  that 
MDRC  monitored  Entitlement. 

MDRC  has  given  Entitlement  sponsors  more  centralized 
direction  and  closer  monitoring  than  Labor  normally  provides 
for  other  CETA  programs.  As  Labor's  guiding  agency  for 
Entitlement,  MDRC  has  certain  operational  responsibilities 
in  addition  to  its  research  duties.  To  guide  and  assess  the 
performance  of  sponsors  from  both  operational  and  research 
standpoints.  Labor  required  that  MDRC  establish  continuous 
communication  with  the  local  Entitlement  projects. 
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Through  its  field  operations  staff  and  onsite  monitors 
at  each  Tier  I  location,  MDRC  monitors  the  Entitlement  proj¬ 
ects  to  see  that  they  fulfill  their  programmatic,  fiscal, 
and  legal  responsibilities.  MDRC  also  monitors  to  see  that 
the  projects  operate  consistently  with  the  overall  objectives 
of  YEDPA  and  the  program  design.  MDRC  field  and  onsite  staff 
prepare  periodic  reports  on  operations  at  each  project. 
Additional  responsibilities  include  the  identification  and 
assessment  of  prime  sponsor  training  and  technical  assistance 
needs.  In  this  regard,  MDRC  is  responsible  for  identifying 
and  recommending  consultants  when  needed  and  channeling  use¬ 
ful  operational  information  to  the  Entitlement  sponsors. 

Program  reviews 

During  the  summer  of  1978,  MDRC  became  aware  of  the  prob¬ 
lems  in  managing  a  large-scale  demonstration.  The  summer 
months  gave  Entitlement  sponsors  their  first  real  operational 
test  because,  in  addition  to  their  Entitlement  responsibili¬ 
ties,  they  were  also  responsible  for  implementing  the  Summer 
Youth  Employment  Program.  In  some  locations,  sponsors  oper¬ 
ated  the  programs  in  an  integrated  fashion. 

MDRC  records  indicate  that  problems  erupted  at  almost  all 
Tier  I  and  some  of  the  larger  Tier  II  locations.  MDRC  re¬ 
sponded  by  sending  teams  to  the  sponsors  to  make  firsthand 
program  reviews.  Typically,  the  teams  consisted  of  Labor 
representatives  and  MDRC  senior  level,  field  operations, 
information  system,  and  onsite  staff. 

The  scope  of  the  review  was  a  comprehensive  walk-through 
of  the  program,  component  by  component,  to  help  the  team 
understand  the  program's  operations.  The  program  reviews 
caused  some  sponsors,  who  were  unaccustomed  to  such  close 
examination,  to  assess  and  redirect  operations.  The  spon¬ 
sors,  Labor,  and  MDRC  agreed  that  such  reviews  were  an  effec¬ 
tive  management  tool.  As  a  result,  program  reviews  became  a 
regular  occurrence  with  a  specified  format.  Each  Tier  I 
sponsor  has  had  at  least  two  reviews  and  over  half  of  the 
Tier  II  sponsors  have  had  reviews. 

MDRC  designed  the  program  reviews  as  a  means  to  assess 
management  and  operational  capability  and  performance,  to 
identify  specific  problem  areas,  and  to  recommend  any  neces¬ 
sary  corrective  measures.  In  addition,  the  reviews  include 
an  assessment  of  the  management  and  operation  of  the  infor¬ 
mation  and  fiscal  systems.  Tier  II  program  reviews  also 


include  an  assessment  of  the  program's  innovative  feature. 

The  reviews  are  comprehensive  and  include  sponsor  presenta¬ 
tions  on  recruitment,  intake,  worksite  development  and  moni¬ 
toring;  monitoring  and  enforcement  of  academic  and  attendance 
standards;  and  payroll,  fiscal,  budget,  and  information 
system  management.  Team  members  review  participants'  files, 
performance  reports,  enrollment  forms,  and  other  information 
system  records.  They  interview  program  staff  and  visit  work¬ 
sites,  intake  centers,  and  alternative  education  programs  or 
schools.  The  team  concludes  the  program  review  by  providing 
the  sponsor  with  feedback  on  findings  and  recommended  correc¬ 
tive  actions.  Often  followup  visits  are  made  to  see  what 
progress  a  sponsor  has  made  as  a  result  of  the  review. 

We  agree  that  these  comprehensive  and  intensive  program 
reviews  are  an  effective  and  needed  management  tool  and  they 
are  indicative  of  the  degree  of  oversight  necessary  to  main¬ 
tain  quality  youth  programs.  Without  the  intense  oversight 
provided  by  these  reviews,  it  is  difficult  to  state  what  the 
fate  of  Entitlement  would  have  been  at  some  locations.  The 
following  examples  illustrate  positive  impacts  of  reviews 
which  may  not  have  resulted  in  a  normal  program  environment. 

— Shortly  after  program  operations  began,  Denver  encoun¬ 
tered  problems  which  persisted.  Early  in  the  program, 
the  use  of  incorrect  income  guidelines  resulted  in 
enrolling  many  ineligible  youths.  Many  errors  on 
enrollment  forms  were  not  corrected  because  it  would 
have  required  reinterviewing  2,000  youths.  The  pro¬ 
gram  received  negative  media  attention  due  to  enroll¬ 
ing  the  ineligible  youths,  alleged  payroll  fraud,  and 
the  few  participants  placed  in  jobs.  In  April  1978, 
only  200  of  1,200  enrolled  youths  were  assigned  to 
jobs.  Original  projected  average  enrollment  was  3,550. 
Staff  turnover  was  high,  including  the  program  director 
position.  A  poor  relationship  between  the  sponsor  and 
city  agencies  affected  program  operations,  particularly 
payroll.  Relationships  between  the  sponsor  and  most 
contractors  who  were  responsible  for  significant  areas 
of  program  operation  deteriorated. 

MDRC ,  with  Labor's  concurrence,  became  directly  in¬ 
volved  in  managing  the  Denver  Entitlement  program. 
Because  the  sponsor  encountered  difficulty  in  hiring 
needed  employees,  MDRC  agreed  to  pay  the  wages  for 
some  staff  to  perform  monitoring,  management  informa¬ 
tion,  and  other  functions.  MDRC  allowed  its  onsite 


monitor  to  work  closely  with  sponsor  staff  in  all 
areas  of  program  activity  and  report  frequently  to 
MDRC  in  New  York.  MDRC  prepared  and  conducted  a 
training  session  for  Denver  administrative  staff. 

To  establish  a  good  solid  program  on  a  smaller  scale, 
Denver,  at  the  insistence  of  MDRC  and  Labor,  stopped 
enrolling  new  participants  on  June  30,  1979.  As  of 
June  1980,  new  enrollments  were  still  prohibited. 

On  the  basis  of  a  January  1980  review,  MDRC  made  major 
program  and  administrative  recommendations  which  Denver 
adopted.  The  sponsor  obtained  new  contractors  to  per¬ 
form  Entitlement  functions  and  established  control  mech¬ 
anisms  for  monitoring  contractor  performance.  A  public 
accounting  firm  assessed  the  payroll  system  and  recom¬ 
mended  improvements.  Although  some  improvements  were 
apparently  made,  MDRC  saw  a  potential  danger  in  the 
sponsor's  continuing  inability  to  maintain  control  over 
payroll  operations  and  management  information.  MDRC 
recognized  the  need  to  continue  watching  the  situation 
carefully  because  of  the  research  need  for  accurate  data 
and  controls  on  data  collection.  In  spite  of  these  con¬ 
cerns,  MDRC  believes  that  operations  have  stabilized 
and  the  program  is  moving  forward,  although  at  a  much 
smaller  level  than  planned  (about  400  paid  participants). 

— Detroit  also  encountered  problems  soon  after  operations 
began.  During  the  early  months  of  the  program,  about 
30  percent  of  the  participants  waited  8  weeks  or  longer 
to  get  paid.  Many  of  these  youths  had  been  assigned 
to  worksites  before  the  data  necessary  to  put  them  in 
a  pay  status  were  prepared.  As  a  result,  the  payroll 
system  rejected  their  timesheets.  By  May  1978,  less 
than  half  of  the  2,200  enrolled  youths  had  been  as¬ 
signed  jobs.  Administrative  staff  circumvented  basic 
program  systems  to  increase  job  assignments.  There¬ 
fore,  many  participants'  files  and  records  were  lost 
or  left  incomplete;  Entitlement  information  system 
problems  arose;  and  relations  between  the  prime  spon¬ 
sor  and  its  major  contractor — the  school  system — 
deteriorated.  Labor  and  MDRC  pressed  for  resolution 
of  these  problems  and  ordered  a  shutdown  from  Septem¬ 
ber  to  November  1978  to  allow  time  for  their  resolu¬ 
tion.  During  this  period  the  participants  were 
laid  off.  The  sponsor  terminated  its  relationship 


with  the  school  system  as  the  managing  agent  and 
assumed  the  role  itself. 

A  public  accounting  firm  audit  of  the  payroll  system 
in  June  1979  resulted  in  many  recommendations,  but 
problems  continued  with  missing,  incomplete,  or  er¬ 
roneous  payroll  data.  In  addition,  Detroit  continued 
to  have  management  information  problems.  Its  data 
showed  a  significant  discrepancy  between  the  number 
of  participants  enrolled  and  actually  working,  pri¬ 
marily  because  the  sponsor  continued  to  list  many 
youths  as  enrolled  when  they  were  no  longer  active  in 
the  program.  At  one  time,  the  discrepancy  exceeded 
3,000  youths.  In  December  1979,  only  50  percent  of 
the  youths  that  the  sponsor  listed  as  enrolled  re¬ 
ceived  a  paycheck.  After  reviewing  700  files,  the 
sponsor  found  that  400  of  these  youths  should  have 
been  terminated?  others  had  been  terminated  but  not 
removed  from  the  payroll;  and  still  others  needed  a 
job  reassignment.  Despite  these  problems,  the  con¬ 
tinued  involvement  of  MDRC  and  the  assignment  of  more 
qualified  staff  resulted  in  significant  program  im¬ 
provements  . 

— Almost  since  program  inception.  Entitlement  in  Dayton 
suffered  from  three  interrelated  problems:  low  enroll¬ 
ment,  an  unsuccessful  on-the-job  training  component 
(which  was  the  program's  innovative  feature),  and  high 
management  costs.  MDRC  provided  technical  assistance 
by  offering  suggestions  and  recommendations  on  how 
Dayton  should  overcome  its  problems.  In  fact,  MDRC 
gave  the  sponsor  and  its  major  contractor  responsi¬ 
bility  for  taking  specific  corrective  actions  with 
deadlines.  MDRC  suggested  new  recruitment  strategies 
and  told  the  sponsor  to  explain  how  a  contract  change 
would  result  in  increased  on-the-job  training  partici¬ 
pation.  MDRC  also  required  a  plan  to  improve  the 
participant/program  management  cost  ratio  from  40/60 
to  the  approved  67/33.  MDRC  allowed  Dayton  to  expand 
its  recruitment  territory.  The  sponsor  believed  this 
was  the  only  way  to  resolve  its  cost  ratio  problem. 
Enrollment  has  now  increased  from  less  than  50  to  over 
200  after  expansion. 


CHAPTER  5 


CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 


After  over  2  years  of  operating  Entitlement,  serious 
questions  exist  as  to  whether  CETA  prime  sponsors  have  the 
capacity  to  effectively  implement  a  program  of  this  magnitude 
in  a  short  time.  Intensive  monitoring  and  oversight  efforts 
were  implemented  by  Labor  and  MDRC  to  deal  with  a  litany  of 
serious  operational  problems.  These  efforts  met  with  mixed 
results.  Some  problems  were  caused  by  rapid  program  startup 
and  were  resolved  during  the  second  year,  other  problems 
were  more  persistent  and  their  solution  more  evasive,  and 
some  new  problems  surfaced  during  the  second  year.  There¬ 
fore,  a  gradual  approach  to  dealing  with  the  youth  unemploy¬ 
ment  problem  will  be  advantageous  in  the  long  run  because 
sponsors  would  have  the  opportunity  to  develop  the  management 
capacity  needed  to  implement  large-scale  programs. 

Operational  problems  were  so  severe  at  some  locations 
that  it  is  difficult  to  tell  how  the  program  would  have  been 
implemented  had  effective  management  been  employed  by  the 
sponsors.  Although  Labor  and  MDRC  established  goals  and 
regulations  to  implement  the  legislation,  sponsors  had  diffi¬ 
culty  complying  with  the  regulations  and  meeting  the  goals. 

The  question  of  Labor  and  MDRC  intervention  and  its  im¬ 
pact  on  the  Entitlement  demonstration  represents  a  dilemma 
in  implementing  programs  of  this  type.  If  the  demonstration 
had  been  allowed  to  proceed  in  the  normal  CETA  environment 
without  intensive  oversight,  the  program  might  have  failed 
because  of  its  severe  operational  difficulties.  On  the  other 
hand,  the  degree  of  monitoring  and  direct  intervention  that 
took  place  shows  what  is  needed  to  operate  a  complex  youth 
employment  program  but  may  not  be  practical  to  implement  from 
a  financial  or  management  standpoint. 

If  Entitlement  is  to  be  extended,  expanded,  or  a  similar 
program  undertaken,  certain  areas  need  intensive  management 
attention . 
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— First,  it  appears  that  tying  jobs  to  meeting  school 
academic  and  attendance  standards  is  a  valid  concept. 
However,  sponsors  and  schoolr  should  agree  on  meaning¬ 
ful  standards,  and  administrative  processes  need  to  be 
established  to  enforce  them  fairly. 

— Second,  relationships  among  sponsors  and  other  organi¬ 
zations  that  help  youths  need  to  be  upgraded.  Prob¬ 
lems  among  these  organizations  now  often  adversely 
affect  youths. 

— Third,  most  worksites  provided  an  adequate  work  ex¬ 
perience  but  more  effort  is  needed  to  improve  supervi¬ 
sion,  participant  counseling,  job  development,  and 
worksite  monitoring. 

— Fourth,  Entitlement  is  the  first  youth  employment  pro¬ 
gram  which  made  extensive  use  of  subsidized  private 
sector  jobs.  To  the  extent  possible,  attention  must 
now  be  directed  to  assuring  that  jobs  subsidized  by 
the  Federal  Government,  particularly  at  or  near 
100  percent,  are  in  addition  to  jobs  that  normally 
would  be  made  available  to  youths  by  these  employers. 

— Fifth,  the  Entitlement  concept  provides  for  a  guaran¬ 
teed  job;  therefore,  if  it  is  expanded  the  availabil¬ 
ity  of  funds  to  pay  participants'  wages  might  depend 
on  controlling  program  management  costs. 

RECOMMENDATION  TO  THE  CONGRESS 


New  programs,  particularly  a  complex  demonstration  such 
as  Entitlement,  inevitably  face  implementation  problems.  If 
Entitlement  is  to  be  extended  or  used  as  the  basis  for  future 
youth  initiatives,  we  believe  that  the  Congress  should  recog¬ 
nize  the  conditions  under  which  the  demonstration  was  imple¬ 
mented.  Some  competitively  selected  sponsors  receiving  inten¬ 
sive  oversight  had  significant  operational  problems.  If  the 
Entitlement  concept  is  to  be  expanded  to  other  sponsors  or  a 
similar  program  adopted,  we  recommend  that  the  Congress  as¬ 
sure  that  it  be  implemented  gradually  to  allow  sponsors  time 
to  develop  the  necessary  management  capacity. 


RECOMMENDATIONS  TO  THE 
SECRETARY  OF  LABOR 


If  Entitlement  is  extended  or  expanded,  or  a  similar  pro¬ 
gram  is  adopted,  we  recommend  that  the  Secretary 

— require  sponsors  to  strictly  enforce  reasonable 
attendance  and  academic  standards  thereby  enhancing 
the  program's  credibility  and  effectiveness  for  both 
participants  and  the  general  public; 

— increase  efforts  to  promote  cooperative  relationships 
between  sponsors,  schools,  and  other  organizations 
serving  youths; 

— help  sponsors  upgrade  worksite  supervision,  job 
development ,  participant  counseling,  and  sponsor 
monitoring; 

— create,  implement,  and  enforce  adequate  controls  to 
prevent  substitution;  and 

— establish  and  enforce  requirements  that  limit  program 
management  costs. 

MDRC  AND  SPONSOR  COMMENTS 
AND  OUR  EVALUATION 


We  requested  comments  from  Labor,  MDRC,  and  the  11  spon¬ 
sors  included  in  our  review.  MDRC's  response  and  responses 
from  the  Albuquerque,  New  Mexico;  Baltimore,  Maryland;  Hills¬ 
borough  County,  New  Hampshire;  Philadelphia,  Pennsylvania; 
and  Mississippi  sponsors  are  included  in  appendixes  IV  through 
IX.  After  the  30-day  comment  period  provided  by  Public  Law 
96-226  expired,  Labor  submitted  comments  which  concurred  with 
all  of  our  recommendations.  Labor's  comments,  which  are  in¬ 
cluded  as  appendix  X,  have  not  been  analyzed  in  this  report. 

MDRC  and  Baltimore  disagreed  with  our  approach  to  this 
study . 

MDRC 


MDRC  commented  that  our  specific  findings  do  not  sub¬ 
stantially  vary  from  many  of  theirs.  MDRC  did,  however, 
express  some  concern  with  our  draft  report.  It  did  not 
believe  that  the  report  sufficiently  distinguished  between 
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the  structure  of  a  demonstration  and  an  operating  program. 

MDRC  believed  we  failed  to  convey  that  the  primary  mission 
of  the  demonstration  was  to  inform  policymakers  about  the 
feasibility  and  implications  of  the  Entitlement  concept  and 
its  component  parts.  MDRC  pointed  out  that  the  demonstra¬ 
tion  was  to  yield  information  on  whether  the  program  should 
be  continued,  expanded,  reworked,  or  dropped  as  well  as  to 
help  in  the  design  of  other  youth  programs. 

We  recognize  that  Entitlement  is  a  demonstration  and 
refer  to  it  as  such  throughout  the  report.  Furthermore, 
we  discuss  Entitlement  as  a  demonstration  extensively  in 
chapters  1  and  4.  We  agree  that  Entitlement  was  intended 
to  provide  information  to  aid  in  the  design  of  other  youth- 
related  efforts,  and  it  is  on  the  basis  of  information 
gathered  during  our  review  that  we  have  made  our  recommen¬ 
dations  to  the  Congress. 

By  placing  too  much  emphasis  on  the  demonstration  aspect 
of  Entitlement,  however,  MDRC  seems  to  lose  sight  of  the  im¬ 
portance  of  the  program's  operational  aspect.  Much  of  the 
evaluation  of  Entitlement  as  a  demonstration  is  dependent  on 
its  operational  successes  and  failures.  For  example,  MDRC 
stated  that  it  is  not  appropriate  to  say  that  the  sponsors 
failed  to  adequately  enforce  educational  standards.  Instead 
MDRC  said  the  focus  should  be  on  what  could  be  achieved  given 
the  vagaries  of  local  educational  standards.  The  enforcement 
of  educational  standards,  however,  was  not  a  matter  of  choice. 
It  was  specifically  provided  for  in  YEDPA.  It  is  clear  that 
sponsors  had  great  difficulty,  or  were  unable,  to  comply  with 
the  requirement  to  enforce  educational  standards.  Thus,  in 
any  future  programs,  more  emphasis  must  be  placed  on  solving 
the  enforcement  problem  if  the  educational  requirements  are 
retained . 

MDRC  also  stated  that  difficulties  in  the  rapid  startup 
should  be  looked  upon  as  a  learning  process  and  that  sponsors 
which  changed  their  management  structures  should  be  credited 
rather  than  criticized  for  their  willingness  and  ability  to 
make  changes.  We  did  not  criticize  sponsors  for  their  will¬ 
ingness  or  ability  to  change  management  structures.  Rather, 
we  reported  the  conditions  which  led  to  the  changes.  The  key 
lesson  learned  from  the  rapid  startup  forms  the  basis  for  our 
recommendation  to  the  Congress. 
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MDRC  did  not  believe  we  recognized  sufficiently  that  a 
basic  purpose  of  the  demonstration  was  to  assess  its  long¬ 
term  employment  and  educational  impact.  It  pointed  out  that, 
depending  on  the  results  of  the  impact  research,  the  problems 
described  in  the  report  may  not  be  significant.  MDRC  also 
stated  that  the  level  of  supervision  and  control  it  provided 
was  necessary  to  assure  that  the  demonstration  yielded  the 
needed  information  regarding  longer  term  impacts. 

We  recognize  the  importance  of  Entitlement's  research 
of  the  impact  on  youths,  which  is  described  in  chapter  4. 
Obviously,  since  data  were  not  yet  available,  we  could  not 
consider  the  results  of  that  research.  However,  we  believe 
that,  regardless  of  the  outcome  of  the  impact  analysis,  im¬ 
proved  operations  will  translate  into  improved  results.  We 
agree  that  MDRC's  high  level  of  supervision  and  control  was 
necessary  to  obtain  the  needed  information  from  Entitlement. 
However,  it  is  not  likely  to  be  replicated  by  Labor  in  an 
operational  rather  than  demonstration  setting. 

MDRC  questioned  our  evaluation  because  it  lacks  stand¬ 
ards  of  performance.  MDRC  stated  that  the  report  is  slanted 
toward  the  negative,  thereby  leaving  the  reader  with  the  im¬ 
pression  of  failure.  First,  we  would  like  to  point  out  that 
both  positive  and  negative  examples  were  included.  Second, 
we  discussed  the  subject  of  standards,  particularly  for  work¬ 
sites,  with  MDRC  officials  at  great  length  during  our  exit 
conference.  They  agreed  that  specific  standards  of  this 
nature  do  not  exist.  In  the  absence  of  standards,  the  cri¬ 
teria  that  we  used  are  spelled  out  in  the  report. 

MDRC  mentioned  the  issue  of  job  development  and  sponsor 
links  with  other  organizations  as  examples  of  unbalanced 
presentations.  We  disagree.  Regarding  job  development,  we 
state  that  the  sponsors  initially  appeared  to  have  developed 
enough  jobs.  Most  Tier  I  sponsors,  however,  had  difficulty 
creating  or  maintaining  an  adequate  number  of  part-time  jobs 
during  the  school  year.  We  also  point  out  the  administrative 
difficulties  some  sponsors  had  in  assigning  youths  to  jobs  in 
a  timely  manner.  We  support  these  statements  with  examples 
from  three  sponsors.  Furthermore,  MDRC  stated  that  Entitle¬ 
ment  found  jobs  for  80,000  youths  in  a  relatively  short  time. 
The  number  of  youths  assigned  to  jobs  at  any  one  time,  how¬ 
ever,  never  exceeded  30,000. 
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Regarding  the  linkage  issue,  we  credit  all  sponsors  we 
visited  with  making  some  progress.  On  the  other  hand,  we 
state  that  a  number  of  important  relationships  were  not 
always  lasting  or  without  conflict.  We  then  describe  some 
of  the  problems,  their  causes,  and  where  appropriate,  the 
action  taken  to  resolve  them. 

MDRC  pointed  out  several  instances  where  it  claims  the 
report  deals  less  than  adequately  with  certain  issues. 

First,  it  stated  that  intensive,  individual  counseling 
was  not  a  demonstration  objective,  and  it  is  unfair  to  imply 
a  failure  to  achieve  this  as  a  program  goal .  We  have  not 
stated  that  intensive,  individual  counseling  was  an  objective 
of  the  demonstration.  We  did  point  out,  however,  that  ac¬ 
cording  to  their  grant  applications,  which  became  part  of  the 
grant  agreements,  all  Tier  I  sponsors  planned  to  provide  at 
least  some  counseling.  We  believe  that  their  plans  to  pro¬ 
vide  counseling  and  the  high  negative  termination  rate  indi¬ 
cate  that  additional  counseling  would  have  been  advantageous 
to  many  youths. 

Second,  MDRC  stated  that  our  use  of  the  term  "program 
management  costs"  is  misleading.  MDRC,  in  its  reply  to  our 
draft  report,  separated  program  management  costs  from  client 
service  costs  and  stated  that  program  management  costs  aver¬ 
aged  less  than  20  percent.  We  used  the  Entitlement  program 
definition  of  program  management  costs  contained  in  official 
MDRC  instructions  to  the  sponsors  which  combined  program  ad¬ 
ministration  with  other  categories,  such  as  client  services 
and  worksite  supervision.  Regardless  of  how  the  costs  are 
classified,  they  are  higher  than  anticipated. 

MDRC  further  stated  that  our  discussion  of  underenroll¬ 
ment  implies  underachievement.  We  do  not  believe  that  under¬ 
enrollment  necessarily  implies  underachievement.  We  simply 
point  out  that  enrollment  was  less  than  the  sponsors  had 
estimated.  Our  discussion  of  the  underenrollment  issue  is 
not  a  criticism  of  MDRC  or  sponsor  operations.  Instead, 
underenrollment  is  identified  as  a  major  cause  of  the  high 
management  cost  rates. 

Third,  MDRC  stated  that  the  section  of  the  report  on 
displacement  is  weak.  MDRC  pointed  out  that  the  report 
stated  that  an  indepth  review  of  this  issue  was  not  conducted, 
and  that  we  concluded  that  substitution  occurred  in  45  percent 
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of  the  private  sector  jobs.  We  did  not  say  that  substitution 
or  displacement  occurred  in  45  percent  of  the  private  sector 
jobs.  We  state  that  Labor  and  the  sponsors  need  to  monitor 
the  payment  of  wage  subsidies  because  about  45  percent  of 
the  private  sector  jobs  were  with  employers  who  normally  em¬ 
ploy  youths . 

MDRC  also  pointed  out  that  a  major  study  of  the  dis¬ 
placement  issue  is  being  conducted  at  the  behest  of  Labor 
and  the  Office  of  Management  and  Budget.  After  receipt  of 
MDRC ' s  comments,  however,  we  learned  that  the  study,  which 
was  being  conducted  by  two  MDRC  contractors,  was  suspended 
because  its  complexity  resulted  in  considerable  cost  overruns 

Fourth,  MDRC  stated  that  we  confused  short-term  data 
problems  of  the  sponsors  with  the  ultimate  reliability  of 
the  data  for  research  and  reporting  purposes.  MDRC  also 
stated  that  we  misinterpreted  the  quality  control  require¬ 
ments,  giving  them  a  negative  implication.  MDRC  included  an 
attachment  to  its  comments  more  fully  describing  its  Entitle¬ 
ment  Information  System.  In  our  discussion  of  recordkeeping, 
we  did  not  confuse  the  sponsors’  data  problems  with  the  ulti¬ 
mate  reliability  of  the  data  after  they  have  gone  through 
MDRC ' s  quality  control  procedures.  We  specifically  state 
that,  in  spite  of  recordkeeping  problems  at  the  sites,  MDRC 
officials  were  confident  that  the  data  were  sufficiently  re¬ 
liable  for  research  purposes.  Although  we  believe  that  we 
have  sufficiently  summarized  MDRC 1 s  Entitlement  Information 
System  and  quality  control  procedures,  we  have  included  MDRC* 
description  in  appendix  IV. 

Sponsors 


Baltimore  stated  that  we  did  not  give  clear  recognition 
to  some  valuabl  lessons  learned  from  the  demonstration. 
First,  Baltimore  stated  that  the  demonstration  was,  in  part, 
designed  to  determine  whether  large  numbers  of  youths  would 
be  willing  to  work  if  given  the  opportunity.  Baltimore  con¬ 
cluded  that  the  demonstration  proved  that  they  would. 

While  we  do  not  disagree,  we  do  believe  that  it  is  sig¬ 
nificant  to  point  out  that  MDRC  estimated  that  enrollment 
levels  in  the  first  year  were  approximately  40  percent  of 
the  total  eligible  population,  and  in  comments  to  us,  MDRC 
estimated  50  percent  participation  in  1980.  This  indicates 
that  a  large  number  of  youths  did  not  take  advantage  of  the 
Entitlement  opportunity. 
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Second,  Baltimore  stated  that  the  demonstration  was 
successful  in  involving  the  private  sector  in  providing  jobs 
for  youths.  We  agree  that  the  demonstration  did  attract  sub¬ 
stantial  private  sector  involvement.  However,  for  much  of 
the  program,  private  sector  jobs  were  100  percent  federally 
subsidized,  and  a  significant  percentage  were  the  type  that 
youths  would  normally  obtain. 

Third,  Baltimore  stated  that  the  demonstration  shows 
that  sponsors  could  develop  the  management  capacity  to 
operate  large-scale  employment  programs.  While  we  agree 
that  some  sponsors  may  have  eventually  developed  the  manage¬ 
ment  capacity  to  operate  large  programs,  others  have  not. 
Chapter  3  discusses  the  operational  problems  at  length. 
Further,  our  recommendation  to  the  Congress  recognizes  that 
sponsors  need  time  and  assistance  to  develop  the  management 
capacity  necessary  to  operate  large-scale,  quality  employ¬ 
ment  programs . 

In  addition  to  its  general  comments,  Baltimore  raised  a 
number  of  specific  points.  Baltimore  stated  that  the  report 
is  quite  unbalanced  and  takes  issue  with  the  technique  of 
stating  a  general  conclusion  and  using  anecdotal  instances 
from  one  or  two  sites  rather  than  specific  data.  Second,  it 
maintained  that  all  sites  had  both  positive  and  negative  ex¬ 
periences,  and  that  we  did  not  maintain  balance  in  selecting 
our  examples.  To  do  so,  Baltimore  suggested,  invites  un¬ 
warranted  generalizations  and  unfavorable  conclusions. 

We  do  not  agree.  Chapter  2  clearly  credits  sponsors 
with  providing  adequate  work  experiences  at  most  worksites. 
Chapter  1  clearly  describes  the  statistical  sampling  techni¬ 
ques  used  to  select  worksites.  Examples  of  worksites  at 
specific  locations  were  chosen  as  representative  of  positive 
and  negative  aspects  of  the  demonstration  as  a  whole.  Wr 
did  not  compare  the  quality  of  worksites  among  sponsors  be¬ 
cause  our  sampling  plan  only  allowed  us  to  project  to  the 
universe  of  Tier  I  worksites. 

We  have  provided  a  proper  balance  among  the  sponsors, 
where  warranted,  in  describing  the  operational  problems  in 
chapter  3  and  the  data  problems  in  chapter  4 .  We  do  not 
believe  we  have  singled  out  any  particular  sponsor  as  the 
Baltimore  comments  appear  to  assert. 


Baltimore  asserted  that  we  failed  to  recognize  the  value 
of  change.  Again,  we  disagree.  We  believe  that  we  recog¬ 
nize  the  value  of  change  where  it  led  to  improvements.  For 
example,  we  report  that  Baltimore  took  corrective  action  to 
remedy  some  worksite  and  paperwork  problems. 

Baltimore  also  stated  that  we  made  a  number  of  factual 
or  interpretational  errors.  Citing  our  discussion  of  ter¬ 
minations,  Baltimore  stated  that  we  failed  to  recognize  job 
placements  as  positive  terminations.  In  our  analysis,  we 
used  MDRC  termination  statistics  which  classify  job  place¬ 
ments  as  a  neutral  outcome  for  Entitlement  participants.  We 
agree  with  MDRC's  position  since  the  immediate  goal  of  En¬ 
titlement  was  not  to  help  youths  move  to  jobs  outside  the 
program  but  to  help  them  obtain  a  high  school  diploma  or 
its  equivalent. 

Baltimore  stated  that  we  attributed  the  fact  that  it 
had  the  lowest  cost  per  participant  to  the  lower  number  of 
hours  worked  per  participant.  Baltimore  claimed  that,  if 
adjustment  is  made  for  the  lower  hours,  attendance  rates  and 
breaks,  it  still  had  the  lowest  cost  per  participant.  While 
other  factors  may  affect  the  cost  per  participant,  we  believe 
the  number  of  hours  is  the  major  factor  in  Baltimore's  cost 
being  lowest.  Furthermore,  Baltimore  provided  no  support 
for  its  statement  that  it  had  the  lowest  cost  per  participant 
after  these  factors  were  considered. 

Baltimore  stated  that  we  have  taken  data  from  several 
MDRC  sources,  misquoted  the  data,  and  rearranged  them  in  a 
manner  that  changed  the  original  MDRC  intent.  We  have  re¬ 
viewed  the  source  documents  relative  to  the  example  Balti¬ 
more's  comments  cited  regarding  its  information  system.  The 
basic  facts  remain  unchanged;  however,  we  have  made  several 
minor  changes  to  give  the  reader  a  better  perspective  of  the 
problem.  While  the  98-percent  error  rate  mentioned  by  Balti¬ 
more  may  be  an  extreme  situation,  the  fact  still  remains  that 
the  accuracy  and  timeliness  of  its  information  system  data 
remained  a  problem  as  late  as  October  1979. 

Philadelphia  stated  it  was  in  general  agreement  with  the 
major  findings  in  the  areas  of  operational  problems,  worksite 
experiences,  and  the  test  of  Entitlement  as  a  guaranteed  job 
program.  Philadelphia  mentioned  that  three  specific  com¬ 
ponents  of  its  Entitlement  program  deserved  replication: 
private  sector  worksite  development,  clerical  skills  enrich¬ 
ment,  and  concentration  of  services  in  a  targeted  area. 
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Although  we  did  not  specifically  review  these  aspects  in 
Philadelphia,  we  agree  that  successful  programs  should  be 
continued  and  expanded . 

Philadelphia,  in  commenting  on  MDRC's  role  in  the  demon¬ 
stration,  stated  that  many  operational  and  administrative 
constraints  that  have  occurred  can  be  attributed  to  MDRC 
and  do  not  occur  between  the  sponsor  and  the  Labor  regional 
office.  Philadelphia  urged  us  to  include  a  section  in  the 
report  on  MDRC's  role  as  the  program  manager.  We  have  de¬ 
scribed  MDRC's  role  in  chapters  1  and  4.  We  understand  why 
the  sponsor  would  have  a  different  relationship  with  the 
Labor  regional  office  than  it  had  with  MDRC  since  MDRC  pro¬ 
vided  Entitlement  sponsors  with  more  centralized  direction 
and  closer  monitoring  than  Labor  normally  provides  for  other 
CETA  programs . 

Mississippi  concurred  with  our  major  findings  and  sup¬ 
ported  our  recommendations  to  the  Congress  and  the  Secretary 
of  Labor.  Mississippi  did  note  a  few  areas  needing  further 
clarification.  First,  Mississippi  stated  that  its  monitoring 
efforts  were  much  more  thorough  than  indicated  in  our  report. 
We  have  expanded  our  discussion  of  Mississippi's  monitoring 
to  recognize  this  effort. 

Mississippi  also  said  that  MDRC’s  concern  in  March  1980 
that  sponsors  were  reluctant  to  terminate  youths  and  were 
slow  to  enforce  standards  did  not  apply  to  its  current  pro¬ 
gram.  Mississippi  said  that  clearly  defined  academic  and 
attendance  were  established  about  6  months  ago  and  have  been 
enforced  since  that  time.  In  our  view,  this  should  enhance 
Mississippi's  program. 

In  reference  to  our  statement  on  the  low  priority  spon¬ 
sors  placed  on  counseling,  Mississippi  stated  that  counselors 
were  restricted  on  the  amount  of  time  that  could  be  spent  with 
participants,  because  the  participants  were  either  in  class  or 
at  work.  It  stated,  however,  that  more  counseling  is  needed 
and  should  be  considered  in  the  design  of  any  related  program. 
We  agree  and  have  recommended  that  efforts  be  made  to  upgrade 
counseling. 

Albuquerque  stated  that  the  report  was  a  fair  represen¬ 
tation  of  Entitlement.  Hillsborough  County  had  no  specific 
comments  on  the  report. 
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RECENT  GAO  REPORTS  ON 
YOUTH  EMPLOYMENT  AND  TRAINING  PROGRAMS 


"Payment  Problems  in  the  Summer  Youth  Employment  Program 
in  New  York  City"  (HRD-77-18,  Feb.  2,  1977). 

"Information  on  the  Summer  Youth  Employment  Program" 
(HRD-77-121 ,  June  27,  1977). 

"Poor  Administration  of  the  1977  Summer  Program  for  Econo¬ 
mically  Disadvantaged  Youth  in  New  York  City"  (HRD-78-123, 
July  26,  1978) . 

"More  Effective  Management  Is  Needed  to  Improve  the  Quality 
of  the  Summer  Youth  Employment  Program"  (HRD-79-45,  Feb.  20, 
1979). 

"Job  Corps  Should  Strengthen  Eligibility  Requirements  and 
Fully  Disclose  Performance"  (HRD-79-60,  July  9,  1979). 

"Department  of  Labor's  Efforts  to  Improve  the  Quality  of 
the  1979  Summer  Youth  Employment  Program"  (HRD-80-39, 

Feb.  5,  1980). 
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PRIME  SPONSORS  SELECTED  FOR  PARTICIPATION 
IN  ENTITLEMENT  DEMONSTRATION 

Tier  I 

a/1 . 

a/2, 
a/ 3 . 
a/4 . 
a/5. 
a/6 . 

a/7. 


Tier  II 

1.  Alachua  County,  Gainesville,  Fla. 

a/2.  City  of  Albuquerque,  Albuquerque,  N.Mex. 

3.  City  of  Berkeley,  Berkeley,  Calif. 

a/4.  Hillsborough  County,  Manchester,  N.H. 

a/5.  Miami  Valley  Manpower  Consortium,  Dayton,  Ohio. 

6.  County  of  Monterey,  Salinas,  Calif. 

7.  New  York  City,  New  York,  N.Y. 

a/8.  City  of  Philadelphia,  Philadelphia,  Pa. 

9.  Steuben  County,  Bath,  N.Y. 

10.  City  of  Syracuse,  Syracuse,  N.Y. 

a/ Included  in  GAO  review. 


Baltimore  Metropolitan  Manpower  Consortium, 
Baltimore,  Md. 

City  of  Boston,  Boston,  Mass. 

City  of  Cincinnati,  Cincinnati,  Ohio. 

City  and  County  of  Denver,  Denver,  Colo. 

City  of  Detroit,  Detroit,  Mich. 
King-Snohomish  Manpower  Consortium,  Seattle, 
Wash. 

Balance  of  State  Mississippi,  Jackson,  Miss. 
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SCHOOL  ACADEMIC  AND  ATTENDANCE  STANDARDS  FOR  1979 


TIER  I 


Sponsor 

Academic 

Attendance 

Baltimore 

H.S.  : 

60  average 

H  .S  .  : 

no  more  than  4  un¬ 
excused  absences 
per  month 

GED: 

satisfactory 

progress 

GED: 

( none ) 

Boston 

H  .  S  .  : 

passing  grades 

H.S.  : 

no  more  than  25 
percent  unexcused 
absences 

GED: 

satisfactory 

progress 

GED: 

( none ) 

Cincinnat  i 

H.S.  : 

"D"  average 

H  .  S  .  : 

no  more  than  25 
percent  unexcused 
absences 

GED: 

sat  is  factory 
progress 

GED: 

( none) 

Denver 

H.S.  : 

satisfactory  perform¬ 
ance  in  at  least  2 
out  of  3  subjects 

H  .  S  .  : 

no  more  than  5  un¬ 
excused  absences 
per  semester 

GED: 

satisfactory 

proqress 

GED: 

( none ) 

Det  roi t 

H  .  S  .  : 

passing  grades  in 

3  subjects 

H.S.  : 

no  more  than  5  un¬ 
excused  absences 
per  semester 

GED: 

satisfactory 

progress 

GED: 

( none ) 

King- 

Snohomish 

H  .  S  .  : 

passing  grades  in 
one  subject  ( "D" 
average ) 

H  .  S  .  : 

varies  with 
each  district 

GED: 

passing  grades  in 

2  subjects 

GED: 

( none ) 

Mississippi 

H.S.  : 

passing  at  least 

2  subjects  (com- 
mitee  review  of 
individual  cases) 

H  .  S  .  : 

varies  with  each 
di strict 

GED: 

satisfactory 

progress 

GED: 

no  more  than  5 
hours  of  unexcused 

absence  from  class 
per  month 
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TIER  II 


Sponsor 

Academic 

At  t  endance 

Alachua  County 

H.S.  : 

"0"  in  at  least 

4  subjects 

H  *S.  : 

Hawthorne:  no  more  than  15 
unexcused  absences  per  semes¬ 
ter.  Eastside:  no  more  than 
5  unexcused  absences  per 
grad i ng  period 

GED: 

satisfactory  progress 

GED : 

( none ) 

Albuquerque 

H  .  S  .  : 

passing  grades  in 

3  subjects 

H  .  S  .  : 

no  more  than  5  unexcused 
absences  per  qua rt er 

GED: 

satisfactory  progress 

GED: 

( none ) 

Berkeley 

H.S.  : 

"C‘*  average 

H.S.  : 

more  than  3  unexcused 
absences  in  a  6-week  period 
results  in  a  conference  with 
counselor 

GED: 

satisfactory  progress 

GED: 

( none ) 

Dayton 

H.S.  : 

passing  grades  in 

4  or  5  subjects 

H.S.  : 

no  more  than  15  percent 
unexcused  absences 

GED: 

sat isfactory 

GED: 

75  percent  attendance 

Hill sborouqh 

H  .S.  : 

(none  recorded) 

H.S.  : 

no  more  than  7  absences 
per  semester 

GED: 

satisfactory  progress 

GED: 

( none ) 

Monterey 

H.S*: 

passing  grades  in 

4  subjects 

H.S.  : 

no  more  than  2  unexcused 
absences  per  semester 

GED: 

satisfactory  progress 

GED: 

must  attend  at  least  4 
hours  per  week 

New  York 

H.S.  : 

65  average  in  at 
least  2  subjects 

H.S.  : 

no  more  than  5  consecutive 
unexcused  absences--no  more 
than  3  discrepancies  between 
school  and  work  attendance 

GED: 

satisfactory  progress 

GED: 

( none ) 

Phi ladelphia 

H  .S  .  : 

"D"  average 

H  .S.  : 

no  more  than  R  absences 
for  report  period 

GED: 

satisfactory  progress 

GED: 

( none ) 

Steuben  County 

H  .S  .  : 

65  average 

H  .  S  .  : 

absence  from  school  means 
not  allowed  to  workktthat  day 

GED: 

satisfactory  progress 

GED: 

must  attend  at  least  < 
hours  per  week 

Syracuse 

H.S. 

passing  grades  in 

80  percent  of  courses 

H.S.  : 

80  percent  attendance 

GED: 

satisfactory  progress 

GED: 

( none ) 

Notest  The  standards  shown  are  for  the  1978-79  school  year,  and  represent  levels  of 
performance  and  attendance  required  of  youths  in  the  Entitlement  Demonstra¬ 
tion.  They  do  not  necessarily  correspond  to  the  standards  for  satisfactory 
performance  applicable  to  all  school  youths. 

Source:  MDRC,  March  1980,  Second  Interim  Report  on  Program  Implementation. 
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MANPOWER 
DEMONSTRA  TION 
RESEARCH 
CORPORATION 

three  park  avenue 
new  york,  newyork  10016 
212/5323200 

Board  of  Direction 
Hi  <  ■iruhrrft 
(  Hartman 
Hoberl  W«n 
\  ice  (  Hairmon 
HicHard  \atHan 
Tretmirer 
Bernard  Andemin 
Anthony  Dounr 
A1  (art  Utdman 
Alan  K  teller 
David  Schulte 
CiUtert  Steiner 
Phyllu  W  allace 
San  Waterman 

W  illiam  J  (.nnlter 
Prendent 


Mr.  Greqory  J.  Ahart 
Director 

Human  Resources  Division 
U.S.  General  Accountinq  Office 
441  G  Street,  N.W. 

Washington,  D.C.  20548 

Dear  Mr.  Ahart: 


September  25,  1980 


This  is  in  reply  to  your  letter  dated  August  27,  1980  in 
which  you  offered  us  an  opportunity  to  respcrtid  formally  to  the 
GAO  draft  report  on  the  Youth  Entitlement  Pilot  Projects.  I 
would  initially  note  that  members  of  the  Board  and  staff  of  MDRC 
have,  on  several  occasions,  had  the  benefit  of  discussing  with 
GAO  officials  the  scope,  qeneral  nature  and  findings  of  this 
report.  As  we  have  stated  previously,  there  are  certain  areas 
in  which  we  disagree  with  the  approach  taken  by  GAO  in  this 
study . 

These  break  down  into  four  primary  issues  of  concern.  I'iist, 
we  believe  that  the  report  does  not  suf f ieiciit  ly  distituiuish  between 
the  structure  of  a  demonstration  and  an  operating  program.  Second, 
we  are  concerned  that  the  rejvort  does  not  effectively  recognize 
the  long  term  goals  of  the  research.  Third,  we  believe  that  the 
reixirt  fails  to  come  to  grips  with  the  issue  of  how  standards 
should  be  applied  in  making  judgments  about  program  effectiveness. 
Last,  we  teel  there  are  several  subst  ntive  areas  raised  in  the 
report  that  are  not  dealt  with  adequately.  I  will  briefly  treat 
each  of  these  issues  in  turn. 

1.  We  believe'  one  significant  problem  is  the  failure  of 
the  report  to  convey  the  fact  that  this  was  a  demonstration  whose 
primary  mission  was  to  inform  policymakers  on  the  feasibility  and 
implications  of  the  overall  Entitlement  concept  and  its  various 
coiu[x>nent  parts.  Perhaps  the  most  cogent  statement  of  the  purposes 
of  Entitlement  appears  in  the  Rejnirt  of  the  House  Education  and 
Labor  Committee  on  the  Youth  Act  of  1980  (H.R.67H),  recently 
approved  by  the  House.  In  making  its  r oeommendnt ions  on  Entitlement, 
tae  Committee  stated: 


66 


APPENDIX  IV 


APPENDIX  IV 


Unlike  many  other  Federal  programs,  the  Committee 
did  not  harbor  any  preconceived  notions  or 
heightened  expectations.  All  the  Committee  desired 
was  enough  information  to  determine  whether  the 
program  should  be  continued,  expanded,  reworked, 
or  dropped  all  together.  The  Committee  hoped  that 
the  research  would  help  in  the  design  in  other  related 
youth  efforts. 

GAO's  work  in  distinction  to  the  above  purpose  generally  treats 
Entitlement  as  an  operating  program  rather  than  a  demonstration,  designed 
to  be  a  learning  experience.  For  example,  GAO  is  critical  of  the  general 
problems  incurred  by  the  sites  in  establishing  and  enforcing  educational 
standards.  This  was,  indeed,  a  problem  for  individual  sites  because 
within  local  education  systems  such  standards  were  either  non-existent  or 
so  vague  and  varied  as  to  be  useless  as  enforcement  tools.  The  matter  of 
educational  standards  is,  in  addition,  traditionally  one  of  local  respon¬ 
sibility  rather  than  Federal  authority.  It  is  therefore  not  appropriate, 
in  our  view,  to  say  that  the  prime  sponsors  "failed"  to  perform  adequately 
on  this  facet  of  the  demonstration.  Rather  the  focus  should  be  on  what 
realistically  could  be  achieved,  given  the  vagaries  of  local  educational 
standards  so  that  policymakers  could  adjust  their  aspirations  and  expec¬ 
tations  accordingly. 

As  another  example,  GAO's  discussion  of  the  prime  sponsors'  difficulties 
in  the  rapid  start-up  of  the  program  should  be  looked  upon  as  a  learning 
process  in  how  to  structure  the  initial  implementation  of  a  large-scale 
program,  not  as  a  criticism  of  the  prime  sponsors  per  se.  In  fact,  the 
willingness  and  ability  of  several  prime  sponsors,  such  as  Baltimore  and 
Detroit,  to  readjust  their  management  structures  in  the  face  of  extreme 
initial  difficulties  could  be  looked  upon  as  a  virtue  of  the  system  rather 
than  a  criticism  of  it. 

2.  GAO's  approach  is  further  troubled  by  its  failure  to  recognize 
sufficiently  that  a  basic  purpose  of  the  demonstration  was  to  assess  the 
impact  of  a  school-conditioned  job  guarantee  on  the  longer-term  employment 
and  educational  experience  of  its  participants.  It  is  important  to  understand 
this  facet  of  the  demonstration's  research  for  two  reasons.  Perceived 
problems  with  specific  aspects  of  the  program  (i.e.,  limited  counseling, 
lower-than-expected  reenrollment  of  drop-outs,  difficulties  with  educational 
standards)  might  not  be  of  sufficient  weight  to  require  changes  in  the 
program's  structure  if  the  overall  impact  on  the  youths  is  significantly 
positive.  In  addition,  to  ensure  a  reasonable  test  of  longer-term  impacts, 
it  was  necessary  to  provide  a  sufficient  level  of  supervision  and  control 
over  a  demonstration  to  assure  that  it  had  a  fair  chance  of  yielding  the 
necessary  information.  While  it  is  therefore  true  that  a  limited  short¬ 
term  demonstration  can  never  be  "real  world”  in  the  sense  that  I  think  GAO 
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is  using  the  term,  it  still  is  surely  one  of  the  best  techniques  we 
possess  to  inform  policy  as  to  what  works  and  what  doesn't  under  reasonably 
realistic  conditions. 

3.  Another  basic  disagreement  with  the  GAO  evaluation  is  the  way 
standards  of  performance  are  applied  by  GAO  in  its  analysis.  This  issue 
becomes  important  because  the  report  implies  standards  of  quality  against 
which  the  various  facets  of  the  program's  operation  are  judged:  GAO  proceeds 
to  find  fault  by  citing  negative  examples  when  its  implicit  standards  are 
not  met.  Therefore,  because  the  tone  of  the  report  and  the  examples  or 
illustrations  used  are  heavily  slanted  toward  the  negative,  the  reader  is 
left  with  a  distinct  impression  of  failure. 

The  question  should  be  asked:  A  failure  compared  to  what?  In  a 
complex  demonstration  which  found  jobs  for  over  80.000  youths  in  a  relatively 
short  period  of  time,  one  can,  no  doubt,  find  examples  to  justify  any  state¬ 
ment.  A  balanced  analytic  approach,  however,  would  call  for  selecting 
examples  not  only  in  the  context  of  overall  program  accompl ishments ,  but 
would  also  ask  what  could  reasonably  be  expected  from  an  employment  proqram 
geared  to  this  population.  For  example,  the  report  cites  an  ongoing 
problem  that  Tier  I  sites  had  in  developing  sufficient  jobs  for  their  parti¬ 
cipants.  while  we  agree  that  there  were  occasional  difficulties  with  job 
development,  on  the  whole  the  ability  of  prime  sponsors  to  create  rapidly 
large  numbers  of  reasonably  high  quality  jobs  seems  to  have  been  a 
program  strength  rather  than  a  program  weakness.  Or,  as  another  example, 
the  report  appears  to  find  fault  with  the  prime  sponsors'  ability  to  work 
with  the  other  community  agencies  and  institutions  and  again  provides 
examples  to  justify  its  findings.  We  would  suggest  that,  on  balance,  the 
prime  sponsors  were  generally  able  to  work  out  fairly  effective  relationships 
with  numerous  local  aqencies  and,  in  fact,  many  of  the  examples  cited  by 
the  report  as  apparent  prime  sponsor  failures  were  actually  due  to  inadequate 
performances  by  the  delegate  agencies.  These  necessitated  that  prime  sponsors 
take  action  to  terminate  or  alter  relationships  in  the  interests  of  sound 
management . 

4.  As  our  last  major  area  of  concern,  we  would  point  out  several 
instances  where  the  report  deals  less  than  adeouatoly  with  issues  that  it 
raises.  These  include  the  following: 

a)  Counsel ing  -  Intensive,  individual  counseling  was  not  a 
demonstration  objective.  While  some  may  argue  that  such  counseling  is 
needed,  it  is  unfair  to  imply  a  failure  to  achieve  this  as  a  program  goal 
when  it  was  not  part  of  the  design. 

b)  Program  Management  Costs  and  Under-l-ni  ol  lment  -  both  ot 
these  terms  as  used  in  the  report  are  misleading.  Proqram  management  costs, 
as  distinct  from  client  services  costs  (as  that  term  is  normally  used  in 
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the  CETA  system),  were  not  particularly  high  in  this  program,  averaging 
less  than  20%  of  total  costs.  We  believe  it  is,  therefore,  not  accurate 
to  cite  such  costs  as  being  high.  Furthermore,  to  characterize  partici¬ 
pant  levels  as  "under-enrollment"  also  implies  under-achievement.  In 
fact,  participation  levels  (now  approximately  50%  of  eligibiles) ,  while 
lower  than  site  estimates,  are  higher  than  the  researchers'  predictions. 
Again,  going  back  to  the  purposes  of  the  demonstration,  the  fact  is  that 
no  one  knew  what  participant  levels  would  be,  and  a  basic  purpose  of  the 
demonstration  was  to  find  this  out.  Once  participant  levels  and  manage¬ 
ment  and  service  needs  became  clearer,  it  was  logical  to  make  adjustments 
based  on  experience  and  within  the  cost  parameters  generally  acceptable 
in  the  CETA  system. 

c)  Displacement  -  This  section  of  the  report  is  particularly 
weak;  it  deals  superficially  with  a  very  complex  issue.  First,  the  report 
states  that  an  in-depth  review  of  the  issues  was  not  conducted,  and  then, 
citing  a  few  negative  examples,  concludes  that  displacement  or  substitution 
occurred  in  45%  of  the  private  sector  jobs,  based  solely  on  the  age  of 
normal  hires.  While  it  is  altogether  possible  that  the  report's  figures 
will  eventually  prove  accurate,  it  is  clear  that  the  level  of  analysis 
carried  out  in  this  study  is  not  sufficient  to  justify  such  sweeping  con¬ 
clusions.  A  major  study  of  this  issue,  being  carried  out  at  the  behest  of 
the  Department  of  Labor  and  the  Office  of  Management  and  Budget,  is 
currently  underway,  and  we  are  hopeful  that  it  will  better  inform  this 
subject. 


d)  Adequacy  of  Data  in  the  Information  System  -  The  report  con¬ 
tinues  to  confuse  short-term  data  problems,  uncovered  by  GAO  investigators 
at  the  sites,  with  the  ultimate  reliability  of  the  data  in  the  Information 
System  used  for  research  and  reporting  purposes.  It  also  misinterprets 
quality  control  requirements,  generally  giving  them  negative  implications 
rather  than  realizing  that  they  are  a  means  for  assuring  data  completeness, 
accuracy  and  timeliness.  Rather  than  go  into  this  issue  in  detail  in  this 
letter,  I  am  attaching  a  fuller  exposition  of  it. 

As  a  final  comment,  I  would  note  that  the  specific  findings  of  GAO  on 
the  experiences  of  the  Entitlement  Demonstration  are  not  substantially  at 
variance  with  many  of  our  own  findings,  which  we  have  issued  publicly  in 
reports  on  the  quality  of  work  and  the  general  implementation  experience  of 
the  demonstration.  Our  view  is  that,  as  a  learning  experience  which  has  and 
will  continue  to  inform  policymakers  on  what  accomplishments  are  possible 
in  this  complex  arena,  the  demonstration  has  been  a  positive  achievement. 

Sincerely 

William  J.  Grinker 

President 
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A'i  r.viv-ii:MT 

THE  F:4T  I "  r.KT  1  '.PC  EMATI OK  FVSTEM 


Kith  fesj.,*ct  to  GAO  *s  rt :  .-flits  on  the  Ent  i  t er.er.t  Information  System 
(EiS)  and  site  data  collection  »i  d  reporting  activities,  th.  re  are  tin.-e 
areas  lesciving  of  -1  ar  i  f  i  at  )•  in:  the  relationship  of  the  Entitlement 
Ir.fot  tuition  Sys*em  to  »he  v  .riots  it  oarch  analyses,  the  re'.  , tin. ship  of 
local  data  systems  to  the  '-'L  RT  I  iformat  ion  Fystem,  and  the  interpretation 
and  piv;.<  station  of  fact. 

It  is  in:  ortant  to  nuke  ‘he  distinction  between  what  C AO  describes  as 
■  lata  problems  j  omit  inn  from  usor  recordkeeping  and  reporting,  and  the 
accuracy  and  reliability  of  the  dita  finally  used  in  impact,  cost  and 
implementation  research  analysis.  First,  data  provided  by  *die  sponsors  have 
absolutely  no  c  annection  with  data  collected  independently  by  the  research 
subcontractor  from  the  field  survey  efforts  in  experimental  and  comparison 
sites.  The  impact  analysis  is  primarily  dependent  on  the  field  survey  data 
collection  activity,  and  it  is  supplemented  by  program  EIS  data  only  after 
intensive  data  verification  and  cleaning  efforts.  The  Entitlement  Information 
System  is  used  extensively  in  the  cost  and  implementation  analyses,  and 
continuous  counter-checks  and  reconciliations  have  been  employed  to  assure  that 
the  dat  i  used  for  analysis  are  of  sufficiently  high  quality  for  research  and 
rejerting  purposes. 

Pioiram  data  submissions  originate  from  various  local  sponsor  units  and 
Mic-ject  to  thite  levels  of  quality  control  by  MDRC  staff.  Fiist,  each 
in  u  -  *  •  nd  report  is  if  ’owed  independent  ly  for  completeness  an’,  validity 
•  i  .•  Tail  first  :e\ol  of  quality  control  ensures  the  g  nor  1 

1 1  I  ty  of  the  documents  into  the  computerised  information  system.. 
soc,  iMiwiii'  i-.i'iiuloi  :j:  ocessi  nq ,  errors  t  elating  to  a  'cam's  ’curie  ir. 

r.ec  •.  :  ^  .c.tunt  with  i ecpect  to  other  documents  of  a  siniier  ty  e  are 

rest  Ivi  1.  This  is  done  by  .  .ireful  review  of  the  source  document,  cy  telephone, 

,i!  b  ;:r  •■‘buti’ii  of  .iit  nt  «T-u“ncrated  error  iepor*>-  to  th.  :  ;x>r.s:  r .  Third, 
an  i  ■  ' -O'.:  •  re.  . .  'illation  ;  .«  ’ 'S  takes  pla-e,  in\*o'  log  nr.-  ,.i\lv  t  re  M  5 
.its  1>  t  i  ..I  ■■  ■  i'  .nd  -telephone  c.il’.-ir.  'ati.  ?i  ,s  :  o  ■  :  s 

o:,s:i'  ;  v  ri.us  'y:  ••,.  .!  -  mi  ro'  i  ’  .•  lit ,  st.it  :s,  v  ,oef  or...  ho  .•  -.-wo.  ked 

-  are  !  -.calty  r  '  ■  tc-t,  b  th  for  an  indi”idual  per  t  Icipant  a-.i  it  the 
ngre-.’-e,  reg.ir.il'  ss  rf  wl.i.h  p.  •or..m  unit  prepared  the  data  and  w’i  h  ype 
v»f  do  umfcr.t  cent  lined  t  he  Inh  irniinn.  Discrepancies  found  thr.iu  ih  t.r.e  .  nlity 
control  piocesf.es  u  e  r>  s-'.ved  by  repeated  contacts  with  local  t.f.t  r  .'ors  md, 
if  needed,  on-site  visit.-,.  All  EIF  .lata  presented  in  MDRC  research  iepor»s  will 
.iave  none  through  '  h>se  levels  of  quality  control  and  will  have  been  fully 
reconciled. 

It  is  also  important  to  understand  the  difference  between  the  centralized 
Information  System  ..nd  17  possibly  unique  local  systems.  Pi  ior  to  Entitlement, 

.  ach  prime  sponsor  had  an  existing  information  system,  with  local  fo>ns, 
systems  and  procedures,  entitlement  imposed  a  second  set  of  foims  and  procedures 
Although  pt  ime  s:*ir:sors  were  encouraged  to  as-jimilate  these  two  systr.is  and  to 
integrate  pi  occ lures  as  much  as  possible,  many  sites  decided  to  retain  both 


70 


APPENDIX  IV 


APPENDIX  IV 


,:e  n.i‘o  *>i.t  i  *  ics.  '••'hen  this  did  w.-ur,  Jit.itor  frioiity  was 
ur  .ally  pi  ,_ed  on  -,ai-.‘  .inina  ‘lie  ><.euia'.y  of  t  he  lojjl  syst.m,  f  i  ;>s,  and 
files.  Mr.'RC  ront  ;  n.i.  ;js  1  y  jc'-iev>i-d  and  reconciled  rmst  data  but  the  prime 
spo  ■  s-ors  were  ovn.- 1  al  iv  more  conc’i  ned  with  keeping  their  current  files 
accurate.  Par  »he  although  all  modi  f  i  vat  ions  resulting  from  data 

rnconci  1  iat  i<">ns  v.-  re  rtovided  to  loe3l  sites,  the  y  may  or  may  not  h  ive  been 
applied  to  the  1  >-al  data  systems,  depending  upon  whether  the  specifi-;  MDkC 
form.;;  were  ut’li/  d  I  rally  and  whether  'he  data  related  ‘o  current  or 
historical  activity.  This  situation  could  easily  have  resulted  in  i:nifi  an 
variance  betw  -r.  *  he-  utility  of  MPSC's  data  and  that  of  the  rime  «•*.!  sors. 
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CITY  □  F  ALBUQUERQUE-BERNALILLO  COUNTY 
OFFICE  OF  COMPREHENSIVE 
EMPLOYMENT  AND  TRAINING 
ADMINISTRATION 

5051  V2QU 

D suii'  51  .  * 

*1  uuljut  WU.J  I  >ilrt  Ml  X  ,t  r :  H  FI, 


September  26 ,  1980 


Mr.  James  Walsh,  Group  Director 
United  States  General  Accounting  Office 
Human  Resources  Division 
Washington  D.C.  20548 

Dear  Mr.  Walsh: 

As  requested  w a  have  reviewed  the  proposed  report 
to  Congress  on  the  Youth  Incentive  Entitlement  Pilot 
Project . 

The  report  is  a  fair  representation  of  Y.l.E.p.p. 

The  favorable  comments  about  the  Albuquerque  project 
are  appreciated  and  we  do  not  have  any  other  comments 
regarding  the  report. 


Sinoerelv^ 

/ Shirley  J.  Harri 


CD/LC 


CC:  Central  Pile 
Or igj  nator 
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CITY  OF  BALTIMORE 

VVIl  I  l AM  IH1N-M  I)  SI  H At  It  K.  Mayor 


MAYOR'S  OFFICE 
OF  MANPOWER  RESOURCES 

MARION  W.  PINES.  Director 

701  St.  Pout  Street.  Suite  500.  Baltimore.  Maryland  21202 


September  22,  1980 


Gregory  J.  Ahart ,  Director 

.Ini  ted  States  Ceneral  Accounting  Office 

Washington,  D.C.  20548 

Dear  Mr.  Ahart: 

I  have  read  with  interest  the  draft  of  your  report  on  the  Youth  Entitlement 
Demonstration  Project.  There  are  a  number  of  specific  comments  that  I  wish  to 
make  on  the  context  of  the  report  itself,  however,  first  I  wish  to  note  several 
of  the  major  purposes  of  the  Demonstration  which  it  is  important  to  keep  in  mind 
as  we  analyze  the  implementation  and  impact  of  Entitlement. 

1.  The  Demonstration  was,  in  part.,  designed  to  determine  whether  there 
were  indeed  large  numbers  of  unemployed  youth  who  would  be  willing 
to  work  if  they  had  the  opportunity. 

2.  The  Demonstration  also  sought  to  determine  whether  a  significant 
involvement  of  the  private  sector  could  be  enlisted  in  a  .jobs 
program  for  youth. 

5.  It  also  sought  to  determine  whether  Prime  Sponsors  could  develop 
the  management  capacity  to  operate  such  large  scale  employment 
programs . 

Perhaps  the  most  significant  fact  about  the  Entitlement  Demonstration  is 
that  the  answer  to  all  three  of  these  questions  is  an  unequivocal  "yes"  ■ 

This  valuable  lesson,  unfortunately,  does  not  come  through  clearly  in  your  report. 

Another  factor  which  must  be  kept  in  mind  when  reviewing  program  implementation 
is  that.,  when  a  large  city  receives  a  sizeable  grant  to  provide  jobs  for  youth,  this 
becomes  widely  publicized.  The  general  public  doesn’t  always  understand  the  complexity 
involved,  or  the  need  for  slow,  careful  planning.  Public  and  political  pressure  to 
"get  started"  is  enormous.  Seldom  in  such  a  case  do  we  have  the  luxury  of  taking 
the  time  really  needed  for  optimom  program  implementation. 

Prior  to  the  visit  from  the  General  Accounting  office,  1  was  told  by  several 
sources  that  we  could  never  win  with  GAO.  However,  1  refused  to  believe  that. 

We  are  proud  of  our  Entitlement  program  here  in  Baltimore,  and  1  was  determined 
to  be  open  and  fully  cooperative.  Staff  were  instructed  to  be  candid  and  to 


,(Yi*i'  '  ’XT'*  \  >  ■. 
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provide  whatever  was  requested.  Some  mistakes  have  been  made,  and  we  were 
willing  to  face  up  to  those  mistakes,  knowing  that  we  had  already  recognized 
most  of  them  and  had  already  moved  to  effect  change.  I  fear,  however,  that 
this  policy  of  openness  may  have  boomeranged.  Let  me  be  more  specific. 

1.  The  report  is  quite  unbalanced.  First,  your  technique  of  stating 

a  general  conclusion  and  supporting  it  through  the  use  of  anecdotal 
instances  from  one  or  two  sites  rather  than  specific  data, invites 
unwarranted  generalizations.  Secondly,  all  sites  had  both  positive 
and  negative  experiences.  Care  should  be  taken  that  balance  is 
maintained  in  your  selection  of  examples,  and  that  no  site  is  picked 
out  for  only  negative  examples.  This  also  invites  unfavorable  con¬ 
clusions  . 

2.  The  report  does  not  recognize  the  value  of  change.  A  Research  and 
Demonstration  Project,  in  a  review  such  as  GAO  conducted,  must  be 
viewed  differently  from  an  ongoing,  operational  program.  Change 
should  not  be  considered  a  negative  thing.  Program  operators  in 

a  Demonstrative  Project  have  a  responsibility  to  make  changes  and 
adjustments  in  operational  systems  in  order  to  develop  the  best 
possible  administrative  structure  for  future  replication.  We,  there¬ 
fore,  suggest  that  creative  response  to  the  need,  for  change  ought  to 
be  viewed  as  praiseworthy  rather  than  be  subject  to  criticism. 

Similarly,  while  criticism  of  poorly  functioning  systems  is  certainly 
valid,  when  those  systems  have  been  replaced  or  modified  raking  a 
significant  improvement  in  management  operations,  such  should  at  least 
be  recognized  in  the  report. 

3.  A  number  of  factual  or  interpretat i ona 1  errors  exist  which  should  be 
corrected.  For  example,  you  state  on  page  21  that  negative  terminations 
exceed  positive  ones.  Yet  the  only  positive  termination  you  recognize 
is  "high  school  graduate".  In  every  other  CKTA  Program,  placement  in  an 
unsubsidized  job  is  considered  a  positive  outcome.  If  you  add  the  11.7% 

of  job  placements  to  the  26.7%  of  high  school  graduates,  the  total  positive 
oulooiiio  i  ale  .s  3G.4'.,  wli  i  el.  .execeus  1  lie  34.1%  m.g..iivc  t  el  m,  callous  . 
Presumably  there  are  other  terminations  classed  as  "neutral"  which  are  in 
reality  positive  as  well. 

Another  example  is  on  page  30‘  While  noting  that  Baltimore  has  the  lowest 
cost  per  participant,  you  attribute  that  to  a  lower  number  of  hours  worked 
by  participants  in  the  Baltimore  program.  However,  if  adjustment  is  made 
for  the  lower  hours,  taking  into  account  attendance  rates  and  breaks, 
Baltimore  still  lias  the  lowest  cost  per  participant. 

4.  A  governmental  agency  preparing  a  report  for  Congress  should  be  extremely 
cautious  of  how  it  handles  quotes  and  materials  taken  out  of  context.  On 
page  45.  m  quoting  MDRC  sources  regarding  the  Baltimore  program,  material 


GAO  note:  Page  references  have  been  changed  to  agree  with 
the  final  report. 
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is  taken  from  several  reports  issued  at  different  times,  is  removed 
from  several  different  contexts,  misquoted  and  rearranged  so  as  to 
convey  a  totally  different  impression  from  t  hat  which  MDRC  intended 
in  any  of  the  reports.  (For  example,  the  "98%  error  rate  for  forms" 
noted  in  this  section  refers  to  one  week  in  July,  during  a  peak 
payroll  period,  and  was  not,  as  one  could  imply,  a  norma)  condition.) 

We  suggest  that  all  such  quotations  and  paraphrases  be  examined  and 
adjusted  to  more  clearly  and  objectively  transmit  the  author's  intent. 

In  conclusion,  we  are  pleased  (Mo  found  that  the  vast  majority  of  Kntitlemcnt 
worksites  fostered  m>od  work  habits,  mur'ided  sufficient  quality  rf  useful 
and  provides  adequate  supervision  for  program  participants.  With  such  results  in 
mind,  it  cannot  be  doubted  that  the  majority  of*  program  participants  received 
valuable  and  meaningful  experiences  during  their  tenure  in  the  program.  We  under¬ 
stand  how  very  difficult  it  must  have  been  for  your  field  staff  to  seek  to  rapidly 
assimilate  and  digest  the  vast  amounts  of  information  collected  regarding  this 
very  complex  program.  We  attempted  to  give  you  every  cooperation  in  this  effort , 
and  we  hope  that  our  comments  upon  your*  first  draft  of  the  report  will  he  beneficial 
to  you  in  further  refining  your  analysis.  We  sincerely  hope  that  this  effort  was 
not  simply  a  warm-up  for  your  next  presentation  on  "(SO  minutes." 

Sincerely,  ^  f 

T&UGm  !i 

Marion  Kt .  1‘iru-s 

Director 


MWt‘:sk 
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Southern  New  Hampshire  Services,  inc. 

EMPLOYMENT  AND  TRAINING  PROGRAM 


P  O  Box  .1041) 

Manchester.  Veu  Hampshire  03!  OS 
Telephone  ftOJ-hhh-0701 


JAMES  A.  MACIUKOS  Office  Routt  114 

CtTA  Adimmitrifm  Goffstown,  Nil  03(145 


September  15,  1980 


Mr.  Gregory  J.  Ahart 
Director.  Human  Resources  Division 
United  States  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Ahart, 

Thank  you  very  much  for  the  opportunity  to  review  the  proposed 
report  to  the  Congress  on  the  Department  of  Labor's  Youth  Incentive 
Entitlement  Pilot  Projects  and  to  comment  on  the  material  in  the 
report  before  it  is  issued  in  final  form. 

At  this  time  the  Hillsborough  County  Prime  Sponsor  will  not  he 
commenting  on  the  proposed  report  and  is  returning  both  drafts  to 
your  office. 


JAM/  1 1 
Enc Insures 
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CITY  OF  PHILADELPHIA 

WILLIAM  J  GREEN 


OFFICE  OF  EMPLOYMENT  AND  TRAINING 
1234  M*r*d  Sired  3rd  Floor 
Phiiadetpnn  P*  19107 


WILUE  F  JOHNSON 

(••r  Du*  to- 


September  24,  1980 


Mr.  Orpgpi-v  Aiiarp 
Director  of  Human  Resources 
Division 

U.S.  General  Accounting  Office 
441  G  Street,  N.W. 

Washington,  D.C.  20548 

Dear  Mr.  Aharg: 

Thark  you  very  much  for  the  opportunity  to  comment 
on  the  draft  of  GAO's  report  on  the  Youth  Incentive 
Entitlement  Pilot  Project.  Philadelphia's  experience, 
generally,  has  been  complex  and  variable  but  specific 
components  of  our  Entitlement  have  shown  considerable 
merit  and  deserve  replication.  Among  them  are: 

1.  Private  sector  worksite  development; 

2.  Clerical  Skills  Enrichment;  and 

3.  Concentration  of  services  in  a  targeted  area. 

Although  these  components  may  not  be  duplicated 
exactly,  variations  are  planned  for  future  programs. 

Unfortunately ,  there  was  no  mention  ol  these  in 
the  draft  report.  We  were  in  general  agreement  with  the 
major  findings  in  the  areas  of  operational  problems, 
worksite  experiences,  and  the  test  of  Y1EPP  as  a  guaranteed 
job  program. 

Regarding  the  issue  of  the  frequent  intervention  of 
Labor  and  its  contractor  in  program  operations,  we  are 
of  the  opinion  that  the  causes  are  worthy  of  further  study. 
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Existing  CETA  delivery  systems  operate  without  inter¬ 
mediaries  such  as  Manpower  Demonstration  Research  Corporation. 
Many  operational  and  administrative  constraints  that  have 
occurred  in  the  Philadelphia  Entitlement  Program  can  be 
directly  attributed  to  this  relationship  and  they  do  not 
occur  in  our  relationship  with  the  Regional  DOL  office. 

We  urge  you  to  seriously  re-evaluate  the  content  of 
the  draft  report  with  intent  to  include  a  section  on  MDRC  '  s 
role  as  the  program  manager. 

Thank  you  very  much  for  your  consideration. 


James  Rowley 
Deputy  Director  of 
Operations 

JR : AH : ar 
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DICK  MOLPUS 

COORDINATOR 

FEDFRAL  ■  STATE  PROGRAMS 


-.V-  ;•  S 

STATE  OF  MISSISSIPPI 


OFFICE  OF  THE  OQVlFNOR 


DR  ROBERT  L  ROBINSON 

EXECUTIVE  DIRECTOR 

JOB  DEVELOPMENT  &  TRAINING 


September  26,  1980 


Mr.  Gregory  J.  Ahart,  Director 
Human  Resources  Division 
United  States  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Ahart: 

Thank  you  for  providing  us  the  opportunity  to  coiranent  on  the  draft 
report  to  the  Congress  on  the  Department  of  Labor’s  Youth  Incentive 
Entitlement  Pilot  Project.  This  office  operates  one  of  the  largestYIEPP 
programs  in  the  nation  and  we  certainly  feel  that  it  has  been  the  most 
successful . 

Staff  has  carefully  reviewed  and  analyzed  this  draft  and  concurs 
with  the  major  findings.  In  addition,  we  fully  support  the  recoimnendations 
detailed  on  pages  53  and  54  for  the  same  ones  have  been  made  at  one  time 
or  another  by  our  monitoring  and  program  staff.  However,  a  few 
discrepancies  or  areas  needing  further  clarif ication  were  noted.  These 
are  listed  below: 

-  The  report  revealed  a  need  for  additional  monitoring  as 
cited  on  page  22  and  included  Mississippi  in  the  list  of 
sites  using  a  monitoring  checklist.  Perhaps  the  GAO  team 
did  not  review  our  monitoring  reports  for  they  are  very 
detailed  in  nature  and  are  not  based  on  single  checklists. 

Monitoring  as  conducted  by  the  Governor's  office  of  Job 
Development  and  Training  was  and  remains  thorough  and 
complies  with  regulatory  requirements. 

-  The  statement  on  paqe  31,  " . sponsors  were 

reluctant  to  terminate  youth  and  were  slow  to  enforce 
standards,"  is  misleading  in  regard  to  our  program. 

Clearly  defined  academic  standards  for  In-School  or 
Alternative  Education  students  and  class  attendance 
standards  for  Alternative  Education  students  were  not 
established  until  the  last  six  months  of  entitlement. 

However,  at  that  point,  the  standards  were  enforced 
and  participants  who  did  not  meet  such  standards  were 
terminated. 


P  O  BOX  22809  •  JACKSON.  MS  3920S  •  (8011  JS4  7676 
"An  Equal  Opportunity  Employar  M/F/H" 
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-  The  report  states  that,  "Sponsors  placed  a  low  priority 
on  counseling."  In  reality,  the  In-School  Entitlement 
Counselors  were  extremely  restricted  on  the  amount  of 
time  which  could  be  spent  with  participants  for  the 
participants  were  either  in  class  or  working.  More 
counseling  is,  indeed,  needed  and  these  difficulties 
should  be  considered  in  designing  any  related  program. 

Again,  we  appreciate  this  opportunity  and  your  consideration  of  our 
comments.  If  you  have  any  questions,  please  contact  Mr.  Herman  Wells  at 
601-354-6155. 


Robert  L.  Robinson 
Executive  Director 


RLR/JD/mg 


GAO  note:  Page  references  have  been  changed  to  agree  with 
the  final  report. 
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U.  S.  Department  of  Labor  inspectot  General 

Washington,  D  C  20210 


OCT  9  1980 


Mr.  Gregory  J.  Ahart 
Director 

Human  Resources  Division 
U.S.  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Ahart: 

This  is  in  reply  to  your  letter  to  Secretary  Ray  Marshall 
requesting  comments  on  the  draft  GAO  report  entitled, 

"A  Major  Department  of  Labor  Demonstration  Program  provides 
a  Number  of  Lessons  on  Implementing  youth  Programs. "  The 
Department's  response  is  enclosed. 

The  Department  appreciates  the  opportunity  to  comment  on 
this  report. 


RONALD  GOLDSTOCK 
Acting  Inspector  General 


Enclosure 
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U.S.  DEPARTMENT  OF  LABOR'S  RESPONSE  TO 
THE  DRAFT  GENERAL  ACCOUNTING  OFFICE  REPORT 
ENTITLED  — 


"A  Major  Department  of  Labor 
Demonstration  Program  provides 
a  Number  of  Lessons  on  Implement i _n£ 
Youth  Programs" 


Recommendation: 


The  Secretary  should  require  sponsors  to  strictly  enforce 
reasonable  attendance  and  academic  standards  thereby  enhancing 
the  program's  credibility  and  effectiveness  for  both  participants 
and  the  general  public. 


Response : 

The  Department  concurs.  t 


Comments^ 

This  requirement  has  been  demanding  ir.  terms  of  time  and  effort 
during  the  demonstration.  There  has  been  a  disjuncture  between 
the  monthly  report  requirements  called  for  in  the  demonstration 
and  the  reporting  patterns  that  exist  in  the  various  school  systems, 
as  well  as  the  problems  of  insufficient  personnel  to  deal  with  both 
data  and  student  needs  on  a  timely  basis.  Although  lengthy 
negotiations  between  the  school  and  prime  sponsors  were  required 
to  establish  the  standards  in  the  first  place,  all  parties  have 
come  to  believe  that  standards  and  attention  to  enforcement  could 
serve  the  youth  and  the  program  better.  However,  time  and 
increasing  cooperation  between  schools  and  prime  sponsors 
suggest  that  the  enforcement  of  standards  is  feasible. 


Recommendation: 


The  Secretary  should  increase  efforts  to  promote  cooperative 
relationships  between  sponsors,  schools  and  other  organizations 
serving  youth. 


Response : 

The  Department  concurs. 


82 


APPENDIX  X 


APPENDIX  X 


Comments : 


These  cooperative  relationships  should  revolve  around  the 
appropriate  functions  of  the  organizations.  The  Entitlement 
Demonstration  has  shown  that  the  various  agencies  did  best 
what  they  had  done  well  before.  In  addition,  the  cooperative 
relationships  can  best  be  maintained  through  a  strong  monitoring 
effort  on  the  part  of  the  Prime  Sponsor  to  ensure  that  the 
guidelines  are  enforced  and  coordination  occurs. 

Recommendation : 


The  Secretary  of  Labor  should  assist  Prime  Sponsors  in  upgrading 
worksite  supervision,  job  development,  participant  counseling, 
and  monitoring. 

Response : 

The  Department  concurs. 


Comments : 


Should  the  Demonstration  be  continued.  Prime  Sponsors  will 
establish  expectations  for  the  workplace  including  the 
supervision,  standards,  job  content,  clarity  of  the  assignment, 
and  value.  The  supervisor  will  be  required  to  be  available  to 
answer  task  related  questions,  provide  continuing  on-the-job 
training,  communicate  performance  expectations,  and  provide 
positive  reinforcement  for  exemplary  work.  The  supervisor 
will  establish  worksite  standards  and  communicate  to  the 
participants  that  adherence  to  standards  is  a  condition  of 
continuing  participation.  Although  there  are  inherent  limits 
to  the  skill  requirements  which  can  be  placed  on  youth  who 
have'  not  yet  completed  high  school  and  who  may  have  need  of 
remedial  education,  job  tasks  and  the  standards  for  those 
tasks  should  meet  the  capabilities  of  the  participants.  When 
the  tasks  to  be  completed  and  the  productivity  expected  are 
clearly  defined,  it  is  easier  for  participants  to  know  when 
they  are  working  well,  when  they  are  meeting  the  expectations 
of  their  supervisors,  and  when  they  have  accomplished  their 
tasks.  In  addition,  the  participants  are  more  likely  to 
perform  better  if  they  believe  that  the  wcrk  makes  a 
contribution  to  the  agency's  goals  and  objectives.  Counseling 
activities  will  be  upgraded  as  the  demonstration  phase  of 
Entitlement  comes  to  an  end  because  as  similar  programs  are 
adopted  in  the  future,  they  will  not  have  the  size  or  magnitude 
of  the  demonstration.  The  GAO  report  implies  that  participants 
received  more  frequent  and  in-depth  counseling  at  smaller 
sized  programs. 
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Recommendation : 


The  Secretary  of  Labor  should  create,  implement,  and  enforce 
adequate  controls  to  prevent  substitution  in  the  workplace. 


Response : 

The  Department  concurs. 


Comments : 

Although  the  true  displacement  effect  will  not  be  known 
until  the  report  is  completed  in  April  1981,  steps  can  be 
taken  now  to  assure  that  minimal  substitution  patterns 
develop  when  using  private  sector  businesses  as  worksites. 

The  first  method  to  curtail  substitution  will  be  to  limit 
the  number  of  private  sector  worksites  that  normally  hire 
youth  without  the  subsidy  such  as  fast  food  establishments. 

With  the  requirements  of  the  proposed  legislation  automat ica 1 ly 
limiting  the  size  ot  Entitlement  programs  in  the  future  it 
will  be  easier  to  develop  an  adequate  number  of  private 
sector  jobs  that  are  not  generally  reserved  for  the  youth 
population.  In  addition,  the  establishment  of  and  adherence 
to  a  reduced  subsidy  level  plan  as  a  condition  for  continuing 
participation  in  Entitlement  will  minimize  the  substitution 
problem. 


Recommendat ion : 


The  Secretary  of  Labor  should  establish  and  enforce  requirements 
that  limit  program  management  costs. 


Response : 

The  Department  concurs. 


Comments : 


In  an  ongoing  operational  mode,  it  would  be  easier  to  establish 
acceptable  management  cost  levels  because  there  is  now  some  data 
from  which  to  draw  information.  Originally,  the  major  cause  of 
the  high  management  cost  rates  was  the  underenrollment  of 
participants.  Experience  from  the  demonstration  will  permit  a 
more  appropriate  estimate  of  the  take  up  rate  for  participation 
and  sponsors  will  better  judge  the  impact  of  competition  from 
other  youth  programs. 
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